How AWS AppSync and Amazon EventBridge unlock real-time data across domains

When AWS announced for AppSync as an EventBrige target, I thought of the use cases this unlocks. For those that aren't aware, it was already possible for AppSync to put an event directly on an event bus. So I figured there was some lambda-less solution that would no be unlocked!

🗒️ I'm not anti-Lambda, I'm pro the-right-tool-for-the-job 😉 If simply passing data around while transforming it, using a Lambda function seem like the wrong tool

👆This post is a mix of the above 2 minute video and the two GitHub repos!

Configuring AppSync to pass data to EventBridge

In hindsight, I may have downplayed the power of EventBridge and AppSync by using localhost:3000 in my demo. In truth, real-time subscriptions can scale beyond 3M events per second.

However the principle is the same: Application 1 is a normal app, doing normal things. Then one day Application 2 decides they want to display that data in real-time. No polling, no sharing of resources, just taking an event and passing along the data associated with it.

It's possible for Application 1 to simple invoke an API endpoint provided by Application 2. But they are coupled. Any changes on either side and there needs to be communication. In an event-driven world, this becomes a non-issue. Application 1 would simply put a message on an event bus. It doesn't know or care what downstream services pick it up.

This scenario is exactly what the first repository does.

As mentioned earlier, AWS AppSync has direct support for Amazon EventBridge as a datasource. That means all the SigV2 signing it handled for you with one line of code:

const eventBridgeDS = api.addEventBridgeDataSource('gameBusDS', bus)

Now this simply forms the connection, however the function that passes the data is also fairly trivial:

export function request(ctx: Context): PutEventsRequest {
    // The data that gets sent to EventBridge
    return {
        operation: 'PutEvents',
        events: [
            {
                source: ctx.stash.eventBridgeSource,
                detailType: ctx.stash.eventBridgeDetailType,
                detail: { ...ctx.prev.result },
            },
        ],
    }
}

export function response(ctx: Context) {
    //Return the data from EventBridge back to AppSync
    return ctx.prev.result
}

Understanding EventBridge rules and targets

Passing an event to an event bus does effectively nothing. Consumers (targets) subscribe to rules on that event bus. The rule looks at the incoming event payload and says, "based on this matching criteria, I will invoke these targets".

From the diagram, we can see that an EventBridge bus has a rule setup that will call an AppSync API.

In the second repo, the code that configures this is as follows:

const mybroadcastRule = new events.CfnRule(scope, 'cfnRule', {
    eventBusName: bus.eventBusName,
    name: 'broadcastToAppSyncRule',
    eventPattern: {
        source: ['game.broadcast'],
        ['detail-type']: ['GameUpdated'],
    },
    targets: [
       //The targets that care about this message
    ]
})

Note the eventPattern object. This rule will invoke the targets if the incoming event payload has game.broadcast as the source and GameUpdated as the detail-type.

The next part of this repo is likely where--if you're like me, you'll make the most small-and-hard-to-detect errors:

Adding the target.

This is because we're using the L1 construct, so there aren't any utilities for better mapping EventBridge data to an AppSync operation. You get intellisense, but as someone who is used to working with L2 constructs primarily, feeling something to be desired.

targets: [
    {
        id: 'appsyncBroadcastReceiver',
        arn: props.appsyncEndpointArn,
        roleArn: ebRuleRole.roleArn,
        appSyncParameters: {
            graphQlOperation: props.graphQlOperation,
        },
        inputTransformer: {
            inputPathsMap: {
                createdAt: '$.detail.createdAt',
                updatedAt: '$.detail.updatedAt',
                name: '$.detail.name',
                homeTeamScore: '$.detail.homeTeamScore',
                awayTeamScore: '$.detail.awayTeamScore',
                currentMessage: '$.detail.currentMessage',
                id: '$.detail.id',
            },
            inputTemplate: JSON.stringify({
                input: {
                    createdAt: '<createdAt>',
                    updatedAt: '<updatedAt>',
                    name: '<name>',
                    homeTeamScore: '<homeTeamScore>',
                    awayTeamScore: '<awayTeamScore>',
                    currentMessage: '<currentMessage>',
                    id: '<id>',
                },
            }),
        },
    },
],

The first 3 lines should make sense. The 4th line is where we pass in the AppSync operation we're working with.

🗒️ Running npx @aws-amplify/cli codegen add in a directory that contains the schema.graphql file will generate the needed types for you.

However, the focus is on the inputPathsMap and the inputTemplate.

Understanding the EventBridge InputPathsMap

As someone who doesn't often work with EventBridge, I'll do my best here.

Essentially, a EventBridge payload is an object of whatever depth. I imagine some are deeply nested. Furthermore, EventBridge doesn't care about what data a target needs. As such, the inputPathsMap is your chance to flatten the data and pull out just the values you need.

Similar to how a Lambda function has event or AppSync has context, EventBridge rules put the data under "$". So the event source is "$.source" while all the arguments that we passed in are under "$.detail".

Understanding the EventBridge InputTemplate

The inputTemplate works in conjunction with the inputPathsMap. If the latter lets us pull out the values we need, then the former allows us to structure it however we like.

Again, an EventBridge rule has no idea how we need our data structured, so it's up to us to tell it.

🗒️ This isn't entirely true. In my testing, when I put in the wrong structure for my AppSync input and tried to deploy, my deploy would fail while performing some validation on my behalf.

It's worth noting that the inputTemplate is a string. It's also good to see that the input object is only there because our AppSync schema has an input field for the publishMsgFromEB Mutation.

Conclusion

These two separate apps form the basis of connecting real-time applications together without them being inherently coupled. Event-driven architectures is a big topic, and I hope this helped you understand just how powerful-yet-approachable it can be when using the AWS CDK to provision your services.

I'm curious to hear your thoughts on this! Let me know in the comments.

Until next time,

Happy Coding 🦦