Outbox Pattern: Make Sure Your Events Don’t Ghost You

Ever built a system where you:

• Save something to the database

• Then fire off an event (e.g. send an email or publish to RabbitMQ)

• And then you pray it doesn’t fail between the DB save and the publish?

Yeah. That’s the problem the Outbox Pattern solves.

Let’s break it down like you’re trying to debug a “why the hell wasn’t that email sent?” issue at 2AM.

---

🤔 The Problem

In a distributed system — say, microservices land — when you perform two actions:

1. Save data to your DB (e.g. a user registered)

2. Publish an event to another system (e.g. send a welcome email)

You run into the dual write problem: if the DB write succeeds but the message broker publish fails, you're toast.

You now have an inconsistent state:

• User is saved ✅

• Email was never sent ❌

And unless you’re logging every failure and retrying, you’ll never know.

---

🧯Enter: The Outbox Pattern

The Outbox Pattern says:

“Don't publish events directly. Instead, save the event alongside your business data in the same DB transaction. Then, use a background process to pick it up and publish it later.”

You're essentially turning your database into a temporary message queue.

---

How It Works (with .NET & EF Core)

1. Save the event in an OutboxMessages table:

public class OutboxMessage
{
    public Guid Id { get; set; }
    public string Type { get; set; }
    public string Content { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime? ProcessedAt { get; set; }
}

---

2. During your business logic:

public class CreateUserCommandHandler : IRequestHandler<CreateUserCommand>
{
    private readonly ApplicationDbContext _db;

    public async Task<Unit> Handle(CreateUserCommand request, CancellationToken ct)
    {
        var user = new User { Email = request.Email };
        _db.Users.Add(user);

        var @event = new UserRegisteredEvent { Email = user.Email };
        var outboxMessage = new OutboxMessage
        {
            Id = Guid.NewGuid(),
            Type = nameof(UserRegisteredEvent),
            Content = JsonSerializer.Serialize(@event),
            CreatedAt = DateTime.UtcNow
        };

        _db.OutboxMessages.Add(outboxMessage);

        await _db.SaveChangesAsync(ct); // One atomic transaction

        return Unit.Value;
    }
}

If SaveChangesAsync fails, nothing is saved. You’re safe.

---

3. Background Worker (Hangfire, HostedService, whatever)

public class OutboxPublisherJob
{
    private readonly IServiceScopeFactory _scopeFactory;
    private readonly IPublisher _publisher;

    public OutboxPublisherJob(IServiceScopeFactory scopeFactory, IPublisher publisher)
    {
        _scopeFactory = scopeFactory;
        _publisher = publisher;
    }

    public async Task ProcessOutbox()
    {
        using var scope = _scopeFactory.CreateScope();
        var db = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();

        var pendingMessages = await db.OutboxMessages
            .Where(x => x.ProcessedAt == null)
            .ToListAsync();

        foreach (var message in pendingMessages)
        {
            var domainEvent = JsonSerializer.Deserialize<UserRegisteredEvent>(message.Content);

            await _publisher.Publish(domainEvent); // MassTransit, MediatR, whatever

            message.ProcessedAt = DateTime.UtcNow;
        }

        await db.SaveChangesAsync();
    }
}

Set this job to run every 5–10 seconds using Hangfire or a background service.

---

Benefits

✅ You get exactly-once event delivery (as far as your DB is concerned)

✅ You never lose events due to message broker flakiness

✅ Your write operations are transactional and safe

✅ You can re-process failed events easily

---

⚠️ Caveats

• You must eventually publish. Your outbox processor better be running.

• Clean-up old processed outbox entries or your DB will look like a Twitter archive.

• Still want idempotent consumers — don’t assume every event is fresh.

---

When Should You Use It?

Use the Outbox Pattern when:

• You're dealing with important, side-effect-causing events (emails, payments, notifications)

• You need consistency between your data and your messages

• You're working in a microservices or event-driven setup

If you're just logging telemetry or debugging info... nah, don't bother.

Let’s talk

Ever lost events in prod? Or built something more robust than this? Drop your tips — I’m always building and always learning.