How to handle asynchronous workloads in .NET using AWS Lambda and SQS

Amazon SQS (Simple Queue Service) provides a robust mechanism for decoupling systems and processing asynchronous workloads. By combining it with AWS Lambda and .NET, you can build scalable, event-driven applications without managing infrastructure.

In this post, you'll learn how to write a .NET Lambda function that processes messages from an SQS queue.

Architecture Overview

An application pushes messages to an SQS queue.
AWS Lambda automatically polls the queue.
When messages arrive, Lambda invokes your function with a batch.
Your function processes each message.

Create a .NET Lambda Project

Start by creating a plain .NET project. You don’t need any special templates for this walkthrough.

dotnet new console -n SqsLambdaProcessor
cd SqsLambdaProcessor

Add the necessary AWS Lambda packages:

dotnet add package Amazon.Lambda.Core
dotnet add package Amazon.Lambda.SQSEvents
dotnet add package Amazon.Lambda.Serialization.SystemTextJson

Rename Program.cs to Function.cs, or create a new Function.cs file, and define your handler:

using Amazon.Lambda.SQSEvents;
using Amazon.Lambda.Core;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

public class Function
{
    public async Task FunctionHandler(SQSEvent evnt, ILambdaContext context)
    {
        foreach (var message in evnt.Records)
        {
            await ProcessMessageAsync(message, context);
        }
    }

    private Task ProcessMessageAsync(SQSEvent.SQSMessage message, ILambdaContext context)
    {
        context.Logger.LogLine($"Processing message {message.MessageId}: {message.Body}");

        // Add your business logic here

        return Task.CompletedTask;
    }
}

Deploy the Lambda

To deploy, you can use the AWS CLI, AWS Console, or the AWS Toolkit for Visual Studio. Here's how to deploy using the AWS CLI with the Lambda .NET tooling:

Package the project:

dotnet lambda package --output-package bin/release/sqs-lambda.zip

Deploy the function:

aws lambda create-function \
  --function-name SqsLambdaProcessor \
  --runtime dotnet6 \
  --handler SqsLambdaProcessor::SqsLambdaProcessor.Function::FunctionHandler \
  --zip-file fileb://bin/release/sqs-lambda.zip \
  --role arn:aws:iam::YOUR_ACCOUNT_ID:role/YOUR_LAMBDA_EXECUTION_ROLE

Replace the role ARN with your actual IAM role that allows Lambda execution and access to SQS.

Connect the SQS Queue

You can configure the SQS queue as an event source:

aws lambda create-event-source-mapping \
  --function-name SqsLambdaProcessor \
  --event-source-arn arn:aws:sqs:us-east-1:123456789012:your-queue-name \
  --batch-size 5

Test Locally

You can write unit tests using the Amazon.Lambda.TestUtilities package:

dotnet add package Amazon.Lambda.TestUtilities

Example test:

[Fact]
public async Task TestHandlerProcessesMessages()
{
    var function = new Function();
    var context = new TestLambdaContext();
    var sqsEvent = new SQSEvent
    {
        Records = new List<SQSEvent.SQSMessage>
        {
            new SQSEvent.SQSMessage { Body = "Test Message" }
        }
    };

    await function.FunctionHandler(sqsEvent, context);
}

Error Handling and Retries

Lambda automatically retries failed messages. To handle failures:

Implement try-catch inside ProcessMessageAsync.
Set up a Dead Letter Queue (DLQ) on your Lambda or SQS.
Use structured logging to track errors.

Best Practices

Make your function idempotent to handle retries safely.
Tune batch size to optimize throughput vs. memory usage.
Keep Lambda’s timeout below the SQS visibility timeout.
Log enough information to troubleshoot failures.

Cleanup

To remove the resources:

aws lambda delete-function --function-name SqsLambdaProcessor
aws sqs delete-queue --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/your-queue-name

Processing SQS Messages with .NET Lambda