Building a notification orchestrator: a decision log

Carl EubanksCarl Eubanks
3 min read

This isn’t a copy‑paste tutorial. It’s a behind‑the‑scenes look at why I chose a specific shape—hexagonal micro‑orchestrator on SNS → SQS—for a cross‑channel notification pipeline, and the trade‑offs I wrestled with along the way.

Problem Statement

Our platform publishes domain action events whenever users have opted in to notifications through certain channels. We needed to convert those raw events into user‑specific notifications (email, SMS, push) without coupling every producer to every delivery channel.

Goal: 1‑second p95 latency, zero duplicate messages, and the ability to bolt on new event types in a week (not a quarter, lol).

Solution Overview


flowchart TD

    %% ───────── Notification pipeline ─────────
    API["API Suite"]
    SNS_TOPIC["AWS SNS Topic"]
    ORCH["Notification Orchestrator"]
    SQS_QUEUE["SQS Notification Queue"]
    NSVC["Notification Service"]
    SNS_DELIVERY["AWS SNS (delivery)"]

    API -->|Publish notification| SNS_TOPIC
    SNS_TOPIC -->|Subscription payload| ORCH

    %% self-loop to show internal enrichment
    ORCH -->|Enrich & fetch user prefs| ORCH

    ORCH -->|Destination-specific payload| SQS_QUEUE
    SQS_QUEUE -->|Consume notification| NSVC
    NSVC -->|Send via channel| SNS_DELIVERY

The orchestrator is a AWS Spring Boot microservice deployed as a container (ECS/Fargate or Kubernetes) that enriches each event with user preferences, then sends it out to an SQS queue to be consumed and fanned out by the notification (micro)service.

Key Architectural Decisions

DecisionWhy I picked itWhat I gave up
SNS → SQS → NotificationService fan outBuilt‑in reliability & inexpensive at our scale; dead‑simple IAM; millisecond latencyregional only
FIFO SQS Queue to Notification microserviceAllows notification service to delegate how to handle the destinations. That is not an orchestrator’s job.Speedy, but may congest due to a little more things the notification service has to do at once. Possibly slows down throughput
Hexagonal (Ports & Adapters)Keeps core “enrich & route” pure; swap SQS for Kafka with a new adapterMore boilerplate wrappers; devs need pattern literacy
Keycloak for user profilesAlready our source of truth; avoids another DBAdds network hop; Keycloak SDK is verbose
Kotlin + Spring for adaptersFirst‑class coroutines, familiar to the teamCold‑start heavier than Node Lambda
Single Orchestrator vs. per‑channel LamdasCentralised logic; easier A/B testingRisk of god‑service—needs guardrails

Trade‑offs & Alternatives Considered

EventBridge Pipes

Pros: Managed transform stage, fewer lines of code.
Cons: At announcement time lacked FIFO target; enrichment limited to Lambda only.

SaaS Routers (Knock, Courier)

Pros: Off‑the‑shelf routing & templates.
Cons: Per‑message cost × our volume → non‑starter; vendor lock‑in.

Full ESB (Kafka + ksqlDB)

Pros: Decoupled replay, SQL‑style transforms.
Cons: Operability tax; overkill for < 10 K msg/min.

DB Triggers + Event Sourcing

Pros: Bypasses need for SNS, low latency consumer api calls
Cons: overkill for <10k/min; reconciliation overhead

Lessons Learned

  • Validate cardinality early. Encapsulating a notification, with a unique account id and embedded destinations allowed us to not bloat our infra with more SQS queues to handle

  • Central isn’t evil—just measured. One orchestrator is fine if it’s ruthlessly scoped to enrichment + routing.

  • Observability before prod. With notifications coming from both the consumer and scheduled events, distributed tracing becomes more necessary. This was setup early since the need for modularity and intention when starting out with microsrevices.

Conclusion

A notification pipeline looks deceptively simple until edge‑cases—missing preferences, duplicate events/notifications, out‑of‑order SMS—bite you. By leaning on AWS primitives and a hexagonal core, I get the flexibility of an old‑school ESB without the bloat—and I can add any consumer with a small PR.

Next up (if there’s interest): updating the api suite to save notification subscription history as an audit trail.

Appendix — Minimal Code & Diagram Fragments

PortResponsibilityAdapter Example
UserManagementPortFetch user profileKeycloakUserManagementServicePortAdapter
NotificationPreferencesRepositoryPortLoad notif prefsNotificationPreferencesRepositoryPortAdapter
NotificationServicePortEmit enhanced requestSqsNotificationServicePortAdapter
0
Subscribe to my newsletter

Read articles from Carl Eubanks directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Carl Eubanks
Carl Eubanks