Azure Service Bus Interview Questions and Answers

Choose Storage Queues if: You need a simple (REST-based GET/PUT/PEEK interface) queue with no particular additional requirements (point-to-point communication). You need an audit trail of all messages that pass through the queue. You expect the queue to exceed 80 GB in size. You want to track progress for processing a message inside of the queue. Choose Service Bus Queues if: You need duplicate detection. You need an at-most-once delivery guarantee.

You need a FIFO/ordering guarantee (using
sessions).

You need to group messages into transactions. You want to receive messages without polling the queue. You need to provide role-based access to the queues. You need to handle messages larger than 64 KB but smaller than 256 KB. Your queue size will not grow larger than 80 GB. You need to have dead-lettering. You would like to be able to publish and consume batches of messages. You want to use a publish-subscribe messaging pattern (using topics and subscribtions).

Peek: This method enables you to view messages without locking or receiving them. Receive: Can work in 2 modes: PeekLock (receives the message but keeps it peek-locked until the receiver abandons the message. The maximum timeout is 5 minutes before message expiration) and ReceiveAndDelete (deletes the message after it is received). So this means you'll use the Peek when your goal is to look at the messages without actually consuming them (maybe you're building a "Queue browser", ... or your process needs to decided if it wants/can consume the message). And you'll use Receive when you're really planning to consume the message and do whatever you need to do with it.

Message scheduling is helpful when you want to postpone non-critical yet essential workloads to a later point in time. Then consumers can pick them up after peak hours. Scheduled messages can be canceled before the enqueue time. To cancel a message, you need its sequence number. For example, you can delay email-sending tasks until midnight to save the computational capacity of consumers. Another case is a consumer writing to an eventually consistent database. Adding a delay will help the message processor catch up with database writes made by upstream components.

Yes. Messages carry a payload and metadata. The metadata is in the form of key-value pair properties and describes the payload, and gives handling instructions to Service Bus and applications. Occasionally, that metadata alone is sufficient to carry the information that the sender wants to communicate to receivers, and the payload remains empty.

Messages once placed in the Dead-Letter sub-queue never expire. They will be available there forever until removed explicitly from the DeadLetter sub-queue. You have to make sure the queues are regularly deleted, possibly in a sensible way (i.e. processed in some way, errors thrown etc.). Also, note that messages expire while dead lettering is disabled (which is the default), which means they'll just get deleted.

Service Bus is for messaging, Event Hub is for event streams, and Event Grid is for event distribution. Azure Event Grid is an eventing backplane that enables event-driven, reactive programming. It uses the publish-subscribe model. Publishers emit events but have no expectations about how the events are handled. Subscribers decide on which events they want to handle. Azure Event Hubs is a big data streaming platform and event ingestion service. It can receive and process millions of events per second. It facilitates the capture, retention, and replay of telemetry and event stream data. Azure Service Bus is a fully managed enterprise message broker with message queues and publish-subscribe topics. The service is intended for enterprise applications that require transactions, ordering, duplicate detection, and instantaneous consistency. Compare:

1. Event Hubs focuses on event streaming where Service Bus is more of a traditional messaging broker.

2. Service Bus is used as the backbone to connect applications running in the cloud to other applications or services and transfers data between them whereas Event Hubs is more concerned about receiving massive volumes of data with high throughput and low latency.

3. Event Hubs decouples multiple event-producers from event-receivers whereas Service Bus aims to decouple applications.

4. Service Bus messaging supports a message property ‘Time to Live’ whereas Event Hubs has a default retention period of 7 days.

5. Service Bus has the concept of a massage session. It allows relating messages based on their session-id property whereas Event Hubs does not.

6. Service Bus The messages are pulled out by the receiver & cannot be processed again whereas Event Hubs messages can be ingested by multiple receivers.

7. Service Bus uses the terminology of queues and topics whereas Event Hubs partitions terminology is used.

8. Event Grids don’t guarantee the order of the events, but Event Hubs use partitions that are ordered sequences, so it can maintain the order of the events in the same partition.

9. Event Hubs are accepting only endpoints for the ingestion of data and they don’t provide a mechanism for sending data back to publishers. On the other hand, Event Grids sends HTTP requests to notify events that happen in publishers.

10. Event Grid can trigger an Azure Function. In the case of Event Hubs, the Azure Function needs to pull and process an event.

11. Event Grids is a distribution system, not a queueing mechanism. If an event is pushed in, it gets pushed out immediately and if it doesn’t get handled, it’s gone forever. Unless we send the undelivered events to a storage account. This process is known as dead-lettering.

12. In Event Hubs the data can be kept for up to seven days and then replayed. This gives us the ability to resume from a certain point or to restart from an older point in time and reprocess events when we need it. Use this loose general rule of thumb: SOMETHING HAS HAPPENED – Use Event Hubs DO SOMETHING or GIVE ME SOMETHING – Use Service Bus

In Azure Service Bus, you can schedule a message to deliver at a later time, but you can also defer a messageuntil a later time. When you schedule a message, you’re telling the Service Bus to deliver that message at a time of your choosing, when you defer a message, you telling the Service Bus to hang onto a message that has been sent, until such time as you’re ready to receive it.

There are scenarios where you'd like to have high throughput and reduced latency and don't care if the store crashes and they are gone (stock updates for example). With express entities, if a message is sent to a queue or topic is, it is not immediately stored in the messaging store. Instead, the message is cached in memory. If a message remains in the queue for more than a few seconds, it is automatically written to stable storage, thus protecting it against loss due to an outage. Writing the message into a memory cache increases throughput and reduces latency because there is no access to stable storage at the time the message is sent. If a message containing critical information that must not be lost is sent to an express entity, the sender can force Service Bus to immediately persist the message to stable storage by setting the ForcePersistence property to true.

The following properties of a queue and topic are immutable. Consider this limitation when you provision your queues or topics, as these properties can't be modified without creating a new replacement entity. Partitioning - enable queues and topics, or messaging entities, to be partitioned across multiple message brokers and messaging stores. Means that the overall throughput of a partitioned entity is no longer limited by the performance of a single message broker or messaging store. Sessions - enable joint and ordered handling of unbounded sequences of related messages. Duplicate detection - helps keep track of the application-controlled MessageId of all messages sent into a queue or topic during a specified time window and queue or topic discards any duplicate copies. Express entity - enable high throughput and reduced latency scenarios. The message is cached in memory.

Yes, it is. You can have multiple listeners in a queue. Let's say we have Process A and Process B that both listen to the queue. A message comes in, and Process A picks it up. If another message comes in and Process A hasn't finished yet, Process B will pick up the message. If Process A is ready, either one of the processes can pick up the second message.

Exactly Once delivery is basically not possible in general case: there's always a chance that the consuming application will crash at the wrong time just before completing the message, and then the message will be re- delivered. You should strive to implement Effectively Once processing. This is usually achieved with an idempotent message processor.

Storing MessageID's (
consumer-side de-duplication) is one option. You could have the policy to clean up old
Message IDs to keep the size of such storage manageable (using sliding time window). To make this 100%

reliable you would have to store Message ID in the same transaction as other modifications done by the processor. Other options really depend on your processing scenario. Find a way to make it idempotent - so that processing the same message multiple times is functionally the same as processing it just once.

With this data delivery semantic, it’s acceptable to deliver a message more than once, but no message should be lost. Use cases: With at-least once, messages won’t be lost but the same message might be delivered (and processed) multiple times. While not ideal from a user perspective, at least once delivery semantics are usually good enough for use cases where data duplication is not a big issue or deduplication is possible on the consumer side. For example, with a unique key in each message, a message can be rejected when writing duplicate data to the database.

SequenceNumber on its own guarantees the queuing order and the extractor order of messages, but not theprocessing order, which requires sessions. If messages just need to be retrieved in order, you don't need to use sessions. If messages need to be processed in order, use sessions. Consider: Say, there are 5 messages in the queue and 2 consumers. Consumer 1 picks up message 1. Consumer 2 picks up message 2. Consumer 2 finishes processing message 2 and picks up message 3 while Consumer 1 is not done with processing message 1 yet. Consumer 2 finishes processing message 3 but consumer 1 is still not done with processing message 1 yet. Finally, consumer 1 completes processing message 1. So, the messages are processed in this order: message 2, message 3, and message 1. If you need message 1, 2, and 3 to be processed in order, you need to use sessions.

SequenceNumber on its own guarantees the queuing order and the extractor order of messages, but not theprocessing order, which requires sessions. If messages just need to be retrieved in order, you don't need to use sessions. If messages need to be processed in order, use sessions. Consider: Say, there are 5 messages in the queue and 2 consumers. Consumer 1 picks up message 1. Consumer 2 picks up message 2. Consumer 2 finishes processing message 2 and picks up message 3 while Consumer 1 is not done with processing message 1 yet. Consumer 2 finishes processing message 3 but consumer 1 is still not done with processing message 1 yet. Finally, consumer 1 completes processing message 1. So, the messages are processed in this order: message 2, message 3, and message 1. If you need message 1, 2, and 3 to be processed in order, you need to use sessions.

At-most-once message processing means that for each message sent, the side effects of its processing may either be missing or applied once. Consider a scenario in which the consumer issues the received request and then crashes before processing it. The Service Bus marks the message as consumed and the application begins consuming messages upon

restart. And the app will miss processing the message that it consumed before the crash. The corresponding receiving mode in ASB is called Receive and delete. In contrast at-least-once message processing means that the side effects may be applied multiple times but they may never be skipped. If the application fails to process the message before the lock timeout expires, Service Bus unlocks the message and makes it available to be received again. If the application crashes after it processes the message, but before it requests the Service Bus service to complete the message, Service Bus redelivers the message to the application when it restarts. Finally, exactly-once message processing means that the side effects are guaranteed to be applied once. Exactly once is the most difficult delivery semantic to implement. It is friendly to users, but it has a high cost for the system’s performance and complexity. Exactly once is especially important when duplication is not acceptable and the downstream service or third party doesn’t support idempotency.

Transactions are execution scopes in the context of transaction processing. A transaction groups two or more operations together. The goal of a transaction is that the result of this group of operations has a common outcome. Azure Service Bus is a transactional message broker and ensures transactional integrity for all internal operations against its message stores and the respective indices. All transfers of messages inside of Service Bus, such as moving messages to a Dead-Letter Queue or automatic forwarding of messages between entities are transactional. From a consumer perspective, Service Bus supports grouping operations against a single messaging entity (queue, topic, subscription) within the scope of a transaction. You can, for instance, send several messages to one Queue from within a transaction scope, and the messages will only be committed to the Queue's log when the transaction successfully completes. The operations that can be performed within a transaction scope are: QueueClient , MessageSender , TopicClient : Send , SendAsync SendBatch , SendBatchAsync BrokeredMessage : Complete , CompleteAsync Abandon , AbandonAsync Deadletter , DeadletterAsync Defer , DeferAsync RenewLock , RenewLockAsync MessageSession : SetState , SetStateAsync GetState , GetStateAsync Transactions that span multiple Service Bus entities require using a special API gesture. You may, for instance, want to receive/complete a message from one queue and send it to a different queue within a single transaction scope. That is done using

transfers ( EnableCrossEntityTransactions ).

A poison message is a message that is sent to the queue or topic that the consuming application cannot process correctly. Both support poison messages but Azure Storage Queue does not support automatic dead-lettering. This means that: in Azure Service Bus you can set expired messages to be moved into a DLQ for later processing, messages with a TTL value that expire in Azure Storage Queue will be deleted permanently.

The Dead-Lettered queue is just another word for Poison queue. So they are the same thing. Azure Service Bus queues and topic subscriptions provide a secondary sub-queue, called a dead-letter

queue (DLQ). The dead-letter queue doesn't need to be explicitly created and can't be deleted or managed

independently of the main entity.

This is not possible out of the box. The only thing you could do is to add an extra filter on every subscription that has something like SubscriptionName='SubscriptionA' . If you then want to send your deadlettered message to the specific subscription, you can add a property SubscriptionName to it, to achieve your goal.

The purpose of the dead-letter queue is to hold messages that can't be delivered to any receiver, or messages that couldn't be processed.

There are several activities in Service Bus that cause messages to get pushed to the DLQ from within the messaging engine itself. For example: The size quota for this stream has been exceeded. The message expired and was dead-lettered. Session-enabled entity doesn't allow a message whose session identifier is null. The maximum number of allowed hops when forwarding between queues has been exceeded. This value is set to 4. Message couldn't be consumed after maximum delivery attempts. An application can also explicitly move messages to the DLQ. They can include messages that can't be properly processed because of any sort of system issue, messages that hold malformed payloads, or messages that fail authentication when some message-level security scheme is used. Once in DLQ messages can be removed from the DLQ and inspected. An application might, with the help of an operator, correct issues and resubmit the message, log the fact that there was an error, and take corrective action.

As the name suggests, **at-most-once **means a message will be delivered not more than once. Messages may be lost but are not redelivered. This is how at-most-once delivery works at a high level. Use cases: It is suitable for use cases like monitoring metrics, where a small amount of data loss is acceptable.

Receive Mode is the way in which messages can be retrieved from a queuing technology by a consumer: Peek & Lease. Used by Azure Storage Queues. The lease duration is associated with the message, so is something that you specify when you send the message to the queue. Once a message is picked up by a consumer, it is not removed from the queue. Instead, a lease (i.e. time-based limit) is placed on the message (on the message level), before it is available for another consumer to process. So in case a service that is holding a resource dies, the resource is freed after the set timeout, and the scenario of deadlock is avoided. Peek & Lock. Used by Azure Service Bus. The message is held in the queue, but a lock is put on the message which is held exclusively by the consumer. The lock can be renewed if needed. By default, this is set at 30 seconds but can be changed. Receive & Delete. Used by Azure Service Bus. The message is considered consumed as soon as the Service Bus has transferred the message (i.e. the consuming application won’t be able to guarantee that the message has been fully processed. So if there’s an issue mid-processing, then the message could be lost). This scenario makes sense if the messages are low-value, but makes little sense in the banking scenario that we outlined earlier.

Both services deal with messages but have very different purposes: Azure Service Bus is an enterprise messaging product. It covers queuing, pub/sub, and has multiple compute-based features. Receiving is done via polling (long polling) and usually, a namespace is accessed within/by a single organization. Azure Service Bus namespace is constrained to a single region. If you need pub/sub within an organization, Service Bus would do. The last Service Bus pros is that there are better event viewers like Service Bus Explorer, whereas there is nothing similar for Event Grid. Azure Event Grid is a notification service. Its sole purpose is to enable pub/sub between event generators and subscribers. It has no queuing semantics. Message delivery is push-based and only a few compute-based features are available unlike with Service Bus. The service is designed to allow communication between multiple parties and can span multiple organizations as published and/or subscribers. Event Grid is global and doesn't have that kind of constraint. Once you need to push notifications about certain events outside of the organization, that's where Event Grid shines. When you have multiple subscribers interested in events that are outside of your trust network, you are not going to hand them out a connection string to your service bus. Rather you'd like to push a notification/message to a webhook that the subscriber will provide. The two can also be mixed. Events from Event Grid can be queued up using Service Bus queues or topics for leveling workloads.

Use Service Bus (Messaging) - For messaging. It’s about one application telling one or more apps to DO SOMETHING or GIVE ME SOMETHING. Receiving is done via polling (long polling) and usually, a namespace is accessed within/by a_ single organization_. Use Event Grid (Eventing) - For eventing when the applications are saying SOMETHING HAS HAPPENED. The service is designed to allow communication between multiple parties and can span multiple organizations as published and/or subscribers. The rule of thumb is: If you can trust the subscribers to connect to the broker, great, use Azure Service Bus. Otherwise, use Even Grid.

There's an important distinction between services that deliver an event and services that deliver a message.

An event is a lightweight notification of a condition or a state change. The publisher of the event has no expectation about how the event is handled. The consumer of the event decides what to do with the notification. Events can be discrete units or part of a series. An event - SOMETHING HAS HAPPENED. An example, an event notifies consumers that a_ file was created_. It may have general information about the file, but it doesn't have the file itself. A message is raw data produced by a service to be consumed or stored elsewhere. The message contains the data that triggered the message pipeline. The publisher of the message has an expectation about how the consumer handles the message. A contract exists between the two sides. A Message - DO SOMETHING or GIVE ME SOMETHING. An example: the publisher sends a message with the raw data, and expects the consumer to create a file from that data and send a response when the work is done.