aws ecs

Architecture Diagram

In this article, let’s walkthrough the process of automating the deployment of a Node.js application to AWS ECS Fargate using an EC2 instance as a Jenkins server, creating a complete CI/CD pipeline to build, test, release, and deploy the application from GitHub whenever code is pushed, thereby streamlining the development and deployment process and making it easier to release updates.

Prerequisites

To automate the deployment of a Node.js application to AWS ECS Fargate using Jenkins on an EC2 instance, here are some prerequisites you'll typically need:

1. EC2 Instance: Set up an EC2 instance where Jenkins will run. This instance should have Java installed (needed for Jenkins) and access to AWS services via IAM roles.

2. Jenkins: Install Jenkins on your EC2 instance.

3. Node.js Application: Have a Node.js application ready that you want to deploy.

4. Docker: Your Node.js application should be Dockerized. This means creating a Dockerfile to package your Node.js app into a Docker image.

5. AWS ECS: Understand the basics of ECS (Elastic Container Service) and Fargate, as you'll be deploying Docker containers to ECS Fargate.

6. IAM Role: Create an IAM role with the necessary permissions for Jenkins to interact with AWS services like ECS.

7. AWS CLI: Install AWS CLI on your Jenkins EC2 instance to interact with AWS services from Jenkins scripts or you can use EC2 connect session manager.

8. Jenkins Plugins: Install necessary Jenkins plugins like AWS Pipeline Plugin, Docker Pipeline Plugin, etc., depending on your pipeline needs.

9. Pipeline Script: Prepare a Jenkins pipeline script (usually written in Groovy) that defines the steps to build your Docker image, push it to a Docker registry (like ECR - Elastic Container Registry), and deploy it to ECS Fargate.

Node.js

. Repository and Configure Git:

- First, clone the GitHub repository:

git clone https://github.com/username/demo.git

- Configure Git with your credentials if not already configured:

 git config --global user.name "Your Name"
     git config --global user.email "your.email@example.com"

2. Build Docker Image:

- Navigate to the cloned repository:

 cd demo

- Build the Docker image using the provided Dockerfile (assuming it's set up correctly):

docker build -t nodejs .

- This command builds an image named nodejs using the Dockerfile in the current directory ('.').

3. Run Docker Container (Locally):

- Start a container using the built image ('nodejs'), mapping port 3000 on the host to port 3000 in the container:

 docker run -d -p 127.0.0.1:3000:3000/tcp nodejs

- This command runs the Docker container in detached mode (`-d`), making it run in the background, accessible at 127.0.0.1:3000.

4. Test Locally:

- Access the Node.js application running inside the container by navigating to http://127.0.0.1:3000 in your web browser.

Initialize Amazon ECR (Elastic Container Registry)

1. Log in to AWS Console:

- Navigate to the AWS Management Console and log in.

2. Create ECR Repository:

- Go to the ECR (Elastic Container Registry) service.

- Click on "Create repository".

Go to Elastic Container Registry service

- Enter a name for your repository (e.g. nodejsonecs).

- Click "Create repository".

Create repository

created ECR

3. Push Docker Image to ECR:

Push commands for nodejsonecs

- Log in to ECR using the Docker CLI:

aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <AWS account ID>.dkr.ecr.<region>.amazonaws.com

Replace <region> and <AWS account ID> with your AWS region and account ID.

- Tag your Docker image with the ECR repository URI:

docker tag nodejsonecs:latest <AWS account ID>.dkr.ecr.<region>.amazonaws.com/nodejsonecs:latest

Replace <AWS account ID>, <region>, and nodejsonecs with your AWS account ID, region, and ECR repository name.

- Push the tagged Docker image to ECR:

docker push <AWS account ID>.dkr.ecr.<region>.amazonaws.com/nodejsonecs:latest

This command uploads your Docker image to the specified ECR repository.

Initialize Amazon ECS

1. Create ECS Cluster:

- Go to the ECS service in the AWS Management Console.

- Click on "Get Started" if it's your first time, or "Create Cluster" if you already have clusters.

Create Cluster

- Choose "Networking only" or "AWS Fargate" as the cluster template.

- Enter a name for your cluster (e.g. nodejsonecsdemo).

- Choose AWS Fargate as the launch type. Keep all other options as default.

- Click "Create".

Select AWS Fargate

2. Creating an ECS Task Definition

1. Navigate to Task Definitions:

- Go to the ECS console

- In the left-hand navigation pane, click on "Task Definitions".

2. Create New Task Definition:

- Click the "Create new Task Definition" button.

Create new Task Definition

3. Select Launch Type Compatibility:

- Choose "Fargate" as the launch type compatibility.

- Click "Next Step".

4. Enter Task Definition Details:

- Task Definition Name: Enter a name for your task definition (e.g., node-server).

- Task Role: Leave it as default unless you have a specific IAM role you need to assign.

- Operating system/Architecture: choose the operating system and CPU architecture for the task. To run your task on a 64-bit ARM architecture, choose Linux/X86_64.

- Network Mode: The default is awsvpc for Fargate, which you should keep.

- Task Execution Role: Leave it as default unless you need to specify a custom role.

Task Definition Details

5. Add Container Definitions:

- Click "Add container".

- Container Name: Enter a name for your container (e.g., nodejs).

- Image: Enter the ECR repository URI for your Docker image. For example,

<aws-account-id>.dkr.ecr.<region>.amazonaws.com/<repository-name>:<tag>

- Memory Limits(Optional): Choose "Soft limit" or "Hard limit" and specify the memory (e.g., 512 MiB).

- CPU Units(Optional): Specify the CPU units (e.g., 256).

- Port Mappings:

- Container port: 3000

- Protocol: tcp

- Environment Variables (Optional): Configure any environment variables required by your application.

- Log Configuration (Optional): Configure logging options if needed.

Add Container Definitions

6. Advanced Container Configuration (Optional):

- Health Check: Set up health checks to ensure your container is running properly.

- Storage and Logging: Configure storage options like mount points and volumes.

- System Controls and Ulimits: Set any system controls or limits if required.

7. Add and Review:

- Once all the required and optional configurations are set, click "Add".

- Review the container definition to ensure all settings are correct.

8. Create Task Definition:

- After reviewing all configurations and ensuring everything is correct, click the "Create" button.

Create task definition

Additional Notes

- Task Role: This IAM role grants permissions to the containers in the task to make AWS API requests.

- Task Execution Role: This role is used by ECS to pull images from ECR and create CloudWatch logs.

- Memory and CPU Settings: Adjust these based on the requirements of your Node.js application.

- Logging Configuration: Useful for debugging and monitoring your container logs via AWS CloudWatch.

3. Creating an ECS Service

Follow these detailed steps to create an ECS Service using the task definition you previously created:

1. Navigate to Your ECS Cluster:

- Go to the ECS console

- In the left-hand navigation pane, click on "Clusters".

- Select your ECS cluster where you want to create the service.

2. Create a New Service:

- Under the "Services" tab, click on "Create".

Create a New Service

3. Configure Service Details:

- Launch Type: Select "Fargate".

- Task Definition: Choose the task definition you created earlier (`node-server`).

- Revision: Select the specific revision of the task definition you want to use (default is the latest).

- Service Name: Enter a name for your service (e.g. nodejs-web-server).

- Number of Tasks: Set the desired number of tasks to run concurrently (e.g., 1).

4. Configure Networking:

- Cluster VPC: Choose the VPC where your service will run.

- Subnets: Select at least two subnets in different availability zones for high availability.

- Security Groups: Choose an existing security group or create a new one. Ensure the security group allows inbound traffic on the port your application uses (e.g., TCP 3000).

Configure Networking

5. Configure Load Balancing (Optional):

Load Balancer Type: Select "Application Load Balancer".
Create a New Load Balancer: Ensure this option is selected.
Load Balancer Name: Assign a name to the load balancer. The name must be alphanumeric and can include hyphens (e.g., nodejs-alb).

Application Load Balancer

Listeners and Routing:

Create a New Listener: Ensure this option is selected.
Listener Port: Set the listener port to 80 for HTTP or 443 for HTTPS.
Create New Target Group: Ensure this option is selected.
Target Group Name: Specify a target group name. The name must be alphanumeric and can include hyphens (e.g. ecs-nodejs-web-server).
Target Group Health Checks: Properly configuring health checks ensures your load balancer routes traffic only to healthy instances of your application.

6. Service Auto Scaling (Optional):

- You can set up service auto-scaling to adjust the number of tasks based on demand.

- Configure scaling policies and thresholds if desired.

7. Review and Create:

- Click "Next".

- Review all your settings to ensure everything is configured correctly.

- Click "Create Service" to deploy your service.

Notes

- Cluster VPC: The VPC must have sufficient IP addresses and be properly configured with internet access if needed.

- Subnets: Using multiple subnets in different availability zones helps achieve high availability.

- Security Groups: Ensure your security group rules allow the necessary inbound and outbound traffic.

- Load Balancer: Setting up a load balancer is optional but recommended for distributing traffic and improving availability.

- Service Auto Scaling: This is useful for handling varying loads and ensuring your service can scale up or down based on demand.

By following these detailed steps, you can successfully create an ECS service that uses the task definition you created earlier, deploys your Node.js application, and configures necessary networking and optional load balancing settings.

Access Your Application:

- Once the service is deployed, access your Node.js application through the DNS name of the load balancer associated with your ECS service.

Validate the CI/CD Pipeline

1. Trigger the Pipeline

- Open Jenkins and navigate to your job.

- Click "Build Now" to manually start the pipeline.

2. Update the Jenkinsfile

- Open the Jenkinsfile in your IDE or text editor.

- Replace the placeholders with actual values:

AWS_ACCOUNT_ID="<YOUR_ACCOUNT_ID>"
AWS_DEFAULT_REGION="<YOUR_REGION>"
CLUSTER_NAME="<YOUR_CLUSTER_NAME>"
SERVICE_NAME="<YOUR_SERVICE_NAME>"
TASK_DEFINITION_NAME="<YOUR_TASK_DEFINITION_NAME>"
IMAGE_REPO_NAME="<YOUR_ECR_REPO_NAME>"
registryCredential = "<YOUR_GIT_CREDENTIAL_NAME>"
JOB_NAME = "<YOUR_JOB_NAME>"

- Commit and push the changes to GitHub.

3. Verify the Pipeline

- Jenkins will detect the changes and run the pipeline automatically.

- Ensure all stages in the pipeline show a successful status in the Jenkins UI.

4. Troubleshoot

- If there are issues, check the console output:

- Click on the build number.

- Select "Console Output".

5. Check ECS Deployment

- Verify that the new task is running in the ECS cluster.

6. Verify Changes in the Application

- Modify a string in the Node.js app, then push the changes to GitHub.

- Check the application via the Application Load Balancer to confirm the changes.

AWS - Deploying a Node.js Application to AWS ECS Fargate with Jenkins Pipeline, Github and Docker