From Idea to Execution: A Step-by-Step Tutorial on Deploying SQS-Polling Lambda Functions Using CloudFormation
Introduction :-
AWS CloudFormation template is designed for deploying a Lambda function that polls an SQS queue for messages on a regular schedule and takes actions accordingly.
The provided CloudFormation template showcases the deployment of a serverless AWS Lambda function, a fundamental building block for event-driven, scalable applications on the AWS platform. This template is written in YAML format and leverages the AWS Serverless Application Model (AWS SAM) for defining the Lambda function and its related resources.
In this template, you’ll find parameters that provide flexibility for customizing the Lambda function deployment according to specific environment and application requirements. Additionally, it configures security groups, integrates with networking resources within a Virtual Private Cloud (VPC), and sets up alarms for monitoring Lambda function performance.
Parameters :-
EnvironmentName
: A string parameter for specifying the environment name.ApplicationName
: A string parameter for specifying the application name.CodeS3Bucket
: A string parameter for specifying the S3 bucket where the Lambda function code artifacts are stored.CodeS3Key
: A string parameter for specifying the S3 key for the Lambda function code artifact.NetworkStackName
: A string parameter for specifying the name of an existing CloudFormation stack that contains networking resources.SNSStackName
: A string parameter for specifying the name of an existing CloudFormation stack that contains SNS resources.SourceSQSAccountId
: A string parameter for specifying the AWS account ID from which SQS messages are replicated.VisitsQueueName
: A string parameter for specifying the name of an SQS queue.
Resources :-
LambdaSecurityGroup
: An EC2 Security Group resource used for the Lambda function. It depends on an imported VPC ID from theNetworkStackName
.SQSPollerLambdaFunction
: The AWS Lambda function resource. It has the following properties:CodeURI: Specifies the location of the Lambda function code.
Handler: Specifies the entry point (handler) for the Lambda function.
MemorySize: The allocated memory for the Lambda function.
Runtime: The runtime environment (Node.js 14.x in this case).
Timeout: The maximum execution time for the Lambda function.
Policies: Defines permissions for the Lambda function to interact with SQS, SNS, and Secrets Manager.
FunctionName: Specifies the name of the Lambda function.
Environment: Specifies environment variables for the Lambda function.
VpcConfig: Configures the Lambda function to run in a VPC with specific subnet and security group settings.
Events: Defines a CloudWatch Events trigger to schedule the Lambda function at a 1-minute interval.
LambdaAlarm
: A CloudWatch Alarm resource that monitors the Lambda function's error rate and triggers an SNS notification if the error rate exceeds a threshold.
Outputs :-
LambdaFunctionConsoleUrl
: An output that provides a URL to the Lambda function's console in the AWS Management Console.
This CloudFormation template is designed for deploying a Lambda function that polls an SQS queue for messages on a regular schedule and takes actions accordingly. It is associated with networking resources (VPC) and SNS topics for monitoring and notifications. Additionally, it sets up CloudWatch Alarms to monitor the Lambda function’s error rate.
Deployment Steps
Follow these steps to upload and create the CloudFormation stack using the AWS Management Console:
- Sign in to the AWS Management Console: Log in to your AWS account if you haven’t already.
2. Navigate to CloudFormation: Go to the AWS CloudFormation service from the AWS Management Console.
3. Click the “Create stack” button.
4. Upload the CloudFormation template file (YAML).
Yaml file :
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless Lambda function
Parameters:
EnvironmentName:
Description: Environment name for the application
Type: String
ConstraintDescription: Specify either dev or production
ApplicationName:
Description: Application Name
Type: String
CodeS3Bucket:
Description: S3 Bucket in which the artifacts are stored
Type: String
MinLength: 3
CodeS3Key:
Description: S3 Key for the lambda artifact
Type: String
MinLength: 3
NetworkStackName:
Description: Name of an active CloudFormation stack of networking resources
Type: String
MinLength: 1
MaxLength: 255
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
SNSStackName:
Description: Name of an active CloudFormation stack of SNS resources
Type: String
MinLength: 1
MaxLength: 255
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
SourceSQSAccountId:
Description: The AWS account from where the SQS messages are to be replicated
Type: String
MinLength: 1
VisitsQueueName:
Description: The AWS account from where the SQS messages are to be replicated
Type: String
MinLength: 1
Mappings: {}
Conditions: {}
Resources:
LambdaSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId:
Fn::ImportValue:
!Sub ${NetworkStackName}-VpcId
SQSPollerLambdaFunction:
Type: AWS::Serverless::Function
DependsOn:
- LambdaSecurityGroup
Properties:
CodeUri:
Bucket: !Ref CodeS3Bucket
Key: !Ref CodeS3Key
Handler: index.sqsPollHander
MemorySize: 128
Runtime: nodejs14.x
Timeout: 30
Policies:
- AWSLambdaVPCAccessExecutionRole
- Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- sqs:ChangeMessageVisibility
- sqs:ChangeMessageVisibilityBatch
- sqs:DeleteMessage
- sqs:DeleteMessageBatch
- sqs:PurgeQueue
- sqs:ReceiveMessage
Resource: !Sub arn:aws:sqs:${AWS::Region}:${SourceSQSAccountId}:*
- Effect: Allow
Action:
- sns:Publish
Resource: !Sub arn:aws:sns:${AWS::Region}:${AWS::AccountId}:*
- Effect: Allow
Action:
- secretsmanager:DescribeSecret
- secretsmanager:GetSecretValue
- secretsmanager:ListSecretVersionIds
- secretsmanager:ListSecrets
Resource: !Sub arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:${EnvironmentName}-secrets*
FunctionName: !Sub ${EnvironmentName}-${ApplicationName}
Environment:
Variables:
NODE_ENV: !Ref EnvironmentName
CARE_EVENT_UPDATES_SNS_TOPIC:
Fn::ImportValue: !Sub ${SNSStackName}-CareEventUpdatesSNSTopicArn
VISITS_QUEUE_URL: !Sub https://sqs.${AWS::Region}.amazonaws.com/${SourceSQSAccountId}/${VisitsQueueName}
VpcConfig:
SubnetIds:
- Fn::ImportValue:
!Sub ${NetworkStackName}-PrivateSubnet1Id
- Fn::ImportValue:
!Sub ${NetworkStackName}-PrivateSubnet2Id
SecurityGroupIds:
- !Ref LambdaSecurityGroup
Events:
SQSPollerEventScheduler:
Type: Schedule
Properties:
Schedule: rate(1 minute)
LambdaAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub ${EnvironmentName}-SQSEventLambdaAlarmForError
AlarmDescription: Alarm if lambda errors out too many times
AlarmActions:
- { 'Fn::ImportValue': !Sub '${SNSStackName}-MonitoringAlarmsSNSTopicArn' }
Namespace: AWS/Lambda
MetricName: Errors
Dimensions:
- Name: FunctionName
Value: !Ref SQSPollerLambdaFunction
Statistic: Average
ComparisonOperator: GreaterThanThreshold
Threshold: '10'
EvaluationPeriods: '5'
Period: '60'
DependsOn:
- SQSPollerLambdaFunction
Outputs:
LambdaFunctionConsoleUrl:
Description: Console URL for the Lambda Function.
Value: !Join
- ''
- - https://
- !Ref AWS::Region
- ".console.aws.amazon.com/lambda/home?region="
- !Ref AWS::Region
- "#/functions/"
- !Ref SQSPollerLambdaFunction
5. Specify Stack Details:
Enter a Stack name for your deployment.
Provide parameter values as needed.
Review and acknowledge the capabilities .
You can set additional stack options or tags if necessary.
6. Review and Create:
Review the stack details and configuration.
Click “Create stack” to initiate the deployment.
7. Monitor Stack Creation:
The CloudFormation stack creation process will begin.
Monitor the stack events in the AWS Management Console.
Conclusion :-
In conclusion, this AWS CloudFormation template offers a structured and automated approach to deploying a serverless AWS Lambda function within your AWS environment. By parameterizing key values like the environment name, application name, and S3 storage location, it provides the flexibility to adapt to various deployment scenarios.
The template’s use of AWS SAM enables a simplified and concise definition of the Lambda function and its dependencies, making it easier to manage and update over time. It also adheres to best practices by configuring security groups, integrating with a Virtual Private Cloud (VPC), and implementing CloudWatch Alarms for proactive monitoring.
Subscribe to my newsletter
Read articles from Mahira Technology Private Limited directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Mahira Technology Private Limited
Mahira Technology Private Limited
A leading tech consulting firm specializing in innovative solutions. Experts in cloud, DevOps, automation, data analytics & more. Trusted technology partner.