Choosing the right Azure Messaging Service for your System

Introduction

Imagine this, you are a solutions architect and you have been tasked with developing a cloud native messaging architecture for your organization's application. Normally, the existing application uses Kafka, RabbitMQ or even ActiveMQ. But now, the organization wants to explore cloud native solutions to take advantage of the resiliency, scalability that the cloud provides. Your organization chooses Azure for this evolution. As an architect you are aware that Azure provides three messaging services: Azure Service Bus, Event Grid and Event Hub. You sit at your desk and are wondering how to choose the best option.

In this article, I explain how you would choose between the three depending on your use case.

First, let’s understand what messaging services are. Messaging services are intermediaries/brokers that sit between a message sender/source and a destination/receiver. Imagine, you are in City A (I love making mental images to make things clearer) and you want to send a message to a friend in City B via a post office. The post office is the broker, the intermediary and relatedly, the messaging service. This is the basic concept.

To choose the best service, as architects, we must understand that Azure messaging services are different for a reason. That is, they send different (or same) messages and use different transport mechanisms to transport these messages based on the message type, size and even the number of messages being sent. Depending on your system’s design, and data flow, some messages need to be received instantly, while others need not to. Thus, it is certain that to determine the best tool, you need to understand what message you are sending.

This means that we need to define:

1. The size of the message.

2. The receiver(s)(who are they and how many).

3. How often you want to send these messages (maybe once or when need occurs).

4. What is the message?

In distributed systems, messages are broken down into 2 categories: Facts and commands based on their intent, direction of flow and how consumers react to them. This distinction helps in designing clearer and more maintainable event-driven or message-driven architecture.

Commands vs Events (Facts)

A command is an imperative message instructing a system to perform an action. It represents an intention to change the state of a system.The sender expects the receiver to acknowledge its reception and process the command. For example, having used a banking application, when you want to transfer funds from your account to another, and hit send. The system initiates the "TransferFunds" command from your account. The "state" of your account and receiver's account is changed (you have lesser money) and the receiver's account has +amount more.

Facts (time series or discrete ) are declarative messages describing a past occurrence and telling another service about it. It's like telling your friend, "Hey the bus already left. Do what you want with this information". At this point it is not your expectation or you don't know or expect how they will react to that information. Ideally, such messages can be sent to single or multiple receivers (1:1 or 1:N respectively). Once sent, these messages are immutable and are append-only. In a banking application (i love fintech :), an event/fact would be likened to a notification to other services for example, Notification Service to send a message, Report Service to generate a statement etc.

Discrete facts are individual self-contained pieces of information that describe an occurrence usually at a specific point in time and have significance to the business domain your system is in. For example, a user logs in a bank application, and you would like to trigger a series of actions such as fraud detection, registering the user, event sourcing such that the system stores every state change as a discrete event for example, AccountDebited, AccountCredited.

I have provided a summary comparison of the two below.

Azure Service Bus

Having understood the difference, now as an architect if you want to send commands, among the three messaging services, you use Azure Service Bus. Why? Azure Service Bus provides reliable delivery (at least once), with queues, or topics/subscriptions, Provides message ordering, dead-lettering and sessions which make it ideal for complex workflows. Also, with the service bus, commands are queued and processed by the dedicated consumer.

Understand that, this is used in considerations:

1. When you have message-based communication between decoupled software systems.

2. When you need to organize and manage distributed transactions in microservices.

3. When you want to provide load balancing the message processing.

4. Where the processing order is vital for your system.

Azure Event Hub

Facts, on the other hand, Time Series Facts/ events usually deal with any telemetry data being relayed as time series such as sensor data. For example, if you have IoT devices that send telemetry data. Imagine that you have an IoT device that sends sensory data every second. In such a system, this is a huge volume of data and you want the system to analyze incoming data and send alerts when certain thresholds are reached. In such a scenario, an event hub is the best choice. Event hub is a big data streaming platform and event ingestion service which can ingest large volumes of data,per second. You can receive these events as a batch and perform real-time batch processing. Therefore, the main uses cases for event hub include:

1. Telemetry data ingestion, aggregation from IoT devices.

2. When you want to perform real-time analytics over the streamed batch data.

3. When you want to understand data archival for anomaly detection.

Imagine you have sensors that send telemetry data such as humidity, temperature, wind direction etc. This sensor will provide real time data every second via Azure Logic Apps APIs. This data is then sent to Azure Event Hub to be processed in real time. We can integrate an Azure Function from which we can send alerts, or even build business intelligence analytics via Azure Data Explorer to create business reports.

Azure Event Grid

When working with discrete or continuous facts, Azure Event Grid becomes the best option. Event Grid is a reactive programming tool that would help you as an architect to design a system which reacts or triggers when specific events happen in a service or another service.

Some of the use cases include:

1. Reacting to status changes in Azure services.

2. Automating business processes and workflows.

3. Integrating with external services or on-premises applications.

Below is a table summarizing the three services and their use cases and suitability for different scenarios.

Conclusion

In conclusion, to effectively implement Azure Messaging Services, you need to first define your messages:

1. Who is the sender and receiver,

2. What is the message size?

3. How many messages are you delivering or receiving?

4. How often are you sending and receiving messages?

5. How soon does the receiver need to process the messages?

After this, understand the capability of each service and do mapping to determine the best service to use.

Coming Up…

In the next article (here), we design a banking transaction and notification service utilizing Azure Service Bus as the message broker. I then provide the Java/Springboot Backend API for these two services. You can find the code in the Github link.

Happy architecting ☺️.