Day 83 of 90 Days of DevOps Challenge: Exploring Amazon SNS and SQS

On Day 82, Yesterday, I explored AWS IAM, the service that defines who can do what with AWS resources. From users and roles to fine-grained JSON policies, IAM showed me how proper access control is essential for both security and seamless operations.

Today, I moved into the world of AWS messaging services: Amazon Simple Notification Service (SNS) and Amazon Simple Queue Service (SQS). These two services are the backbone of event-driven architectures, enabling applications to communicate in a reliable, scalable, and decoupled way.

What is Amazon SNS?

Amazon SNS (Simple Notification Service) is a fully managed messaging service from AWS that enables applications, microservices, and users to communicate with each other through messages (notifications).

SNS is a pub-sub (publish/subscribe) system:

• Publishers → Send messages (like events, alerts, updates).

• Subscribers → Receive those messages (via multiple channels).

Core Components

• Topic → A communication channel that groups subscribers.

• Publisher → The producer of messages (can be an app, AWS service, or user).

• Subscriber → The endpoint that receives messages (e.g., SQS, Lambda, email).

• Message → The actual content sent from publisher to subscriber.

Key Features

• Pub/Sub Messaging → One-to-many messaging; a single message can be sent to many subscribers.

• Multiple Protocols → Supports HTTP/HTTPS, Email, SMS, Mobile Push Notifications, SQS queues, and Lambda functions.

• Scalable & Fully Managed → No servers to manage; automatically scales with workload.

• Durability → Messages are stored redundantly across multiple Availability Zones.

• Filtering → Subscribers can filter which messages they want to receive.

Common Use Cases

• Application alerts via email/SMS.

• Fan-out architecture (e.g., one event → Lambda + SQS + Email).

• Mobile push notifications.

• Decoupling microservices.

Example Flow:

A CloudWatch alarm detects high CPU usage → publishes a notification to an SNS topic → SNS fans it out to:

• An email alert for the ops team.

• An SQS queue for logging.

• A Lambda function that auto-scales resources.

What is Amazon SQS?

Amazon SQS is a fully managed message queuing service from AWS.
It allows applications, services, and microservices to communicate asynchronously by exchanging messages through a queue.

It is like a waiting line (queue) in front of a service counter:

• Producers (applications) send messages to the queue.

• Consumers (applications/services) pick up messages from the queue at their own pace.

This way, even if the consumer is busy or temporarily unavailable, messages won’t be lost.

Two Queue Types

1. Standard Queue

   • Nearly unlimited throughput.

   • At least once delivery (a message may be delivered more than once).

   • Messages might not arrive in exact order.

2. FIFO Queue (First-In-First-Out)

   • Ensures messages are delivered exactly once.

   • Maintains the order of messages.

   • Limited throughput compared to Standard queues.

Core Components

• Queue → A temporary message holding area.

• Producer → Sends messages to the queue.

• Consumer → Reads and processes messages from the queue.

• Message → The data being transferred.

Key Features

• Fully Managed → No servers to run or manage.

• Scalable → Handles millions of messages per second.

• Durable → Messages are stored redundantly across multiple Availability Zones.

• Asynchronous Communication → Producers and consumers don’t need to be connected at the same time.

• Security → Integrated with IAM policies, encryption (KMS), and VPC endpoints.

Common Use Cases

• Order processing systems

• Background job handling

• Decoupling microservices

• Buffering traffic spikes before processing

Example Flow:

• A user uploads a photo to S3.

• An event triggers a message into an SQS queue.

• A worker application (EC2/Lambda) consumes the message, processes the photo, and stores the results back in S3.

SNS vs. SQS

How SNS and SQS Work Together

SNS and SQS are often combined to build scalable event-driven systems:

• SNS → Broadcasts messages (pub/sub model).

• SQS → Stores messages reliably until processed (queue model).

SNS can fan out messages to multiple SQS queues, allowing each queue to be consumed independently.

Real-World Use Case

Imagine an e-commerce platform:

A customer places an order → The order service publishes an Order Placed event to SNS.
SNS fans out the message to different SQS queues:

• Payment Service Queue → Handles payment.

• Inventory Service Queue → Updates stock.

• Email Service Queue → Sends confirmation.

Each service works independently, consuming messages from its queue.

Final Thoughts

At first glance, SNS and SQS may seem simple, but together they solve one of the hardest problems in distributed systems: reliable, decoupled communication.

• SNS → Best for instant broadcast notifications.

• SQS → Best for reliable message processing at your own pace.

• Together → A powerful foundation for scalable, fault-tolerant, event-driven architectures.

Tomorrow, I’ll explore AWS Elastic Beanstalk, AWS’s platform-as-a-service for deploying and managing applications without worrying about infrastructure.