Retrieving Lambda@Edge CloudWatch Logs

What is Lambda@Edge

AWS Lambda@Edge is an extension of the traditional AWS Lambda service, but with a crucial twist – it brings serverless computing capabilities closer to the end-users.

In essence, Lambda@Edge empowers developers to run custom code in response to specific CloudFront events. Whether it's tailoring content based on user location, device type, or handling real-time data processing at the edge, Lambda@Edge functions open up a world of possibilities for optimizing content delivery.

Lambda@Edge functions have the following features:

• They can access public internet

• They can be run before or after your cache

• They allow developers to view and modify not only the client request/response but also the origin request/response

• They can read the body of the request

• They can use a much bigger package size for code compared to CloudFront Functions (1MB for a client request/response trigger and 50MB for an origin request/response trigger)

• They allow up to 5 seconds for a client request/response trigger and up to 30 seconds for an origin request/response trigger.

How can you retrieve the Lambda@Edge logs?

Due to the fact that Lambda@Edge functions run in the CloudFront Regional Edge Caches, the CloudWatch logs cannot only be found in us-east-1, but potentially in all other AWS regions that support Regional Edge Caches.

The Lambda@Edge testing and debugging guide states

When you review CloudWatch log files or metrics when you're troubleshooting errors, be aware that they are displayed or stored in the Region closest to the location where the function executed. So, if you have a website or web application with users in the United Kingdom, and you have a Lambda function associated with your distribution, for example, you must change the Region to view the CloudWatch metrics or log files for the London AWS Region.

So, one simple solution to automatically get the logs from all regions would be to write a script that searches for relevant CloudWatch Logs groups in each region, and displays their respective LogStream entries.

This script could look like this

#!/bin/bash

FUNCTION_NAME=$1
for region in $(aws --output text ec2 describe-regions | cut -f 4) 
do
  echo "Checking $region"
  for loggroup in $(aws --output text logs describe-log-groups --log-group-prefix "/aws/lambda/us-east-1.$FUNCTION_NAME" --region $region --query 'logGroups[].logGroupName')
  do
    echo "Found '$loggroup' in region $region"
    for logstream in $(aws --output text logs describe-log-streams --log-group-name $loggroup --region $region --query 'logStreams[].logStreamName')
    do
      aws --output text logs get-log-events --log-group-name $loggroup --region $region --log-stream-name $logstream | cat
    done
  done
done

If you save this script as cf-logs.sh, after giving it execution rights with chmod +x cf-logs.sh, can then be started with ./cf-logs.sh YOUR_FUNCTION_NAME, where YOUR_FUNCTION_NAME is the real Lambda@Edge function name.

This solution is great for quick log viewing while you're still developing your edge-enabled application, but surely is not a sustainable one when running in production.

Aggregating Lambda@Edge logs with Kinesis

For a production setup, you'd probably want to be able to aggregate and store the individual logs coming from the different regions in one place. A possible solution is to stream them to a Kinesis Firehose Delivery Stream, which then stores the logs in S3.

An example implementation can be found at https://gist.github.com/heitorlessa/5d2295655f9d76483969d215986e53b0

Please be aware that this will incur additional fixed and variable costs, so please review AWS' pricing of the used services.