Streamlining CI/CD on AWS: Mastering Jenkins and Docker Agents (Part 1)

Introduction: Why Jenkins for CI/CD?

Jenkins is a pioneering tool in the realm of Continuous Integration and Deployment (CI/CD), widely adopted for its robust capabilities. It excels in hybrid environments and offers an extensive array of plugins, surpassing tools like GitHub Actions. Despite the emergence of newer tools, Jenkins continues to be a preferred choice for enterprise CI/CD pipelines.

Understanding Jenkins Architecture

Before we install Jenkins, let’s understand how it works:

1. Master Node (Controller Node): Manages the jobs and schedules tasks.

2. Worker Node (Agent Node): Executes the actual build process.

Typically, multiple Worker nodes are used to handle traffic efficiently.

Why Use Docker for Jenkins Agents?

• Traditionally, Worker nodes require separate VMs, but this increases cost and resource usage.

• Instead, we use Docker agents, which:
  ✅ Spin up a container for the build.
  ✅ Auto-delete once done (no extra cost).
  ✅ Are lightweight and fast compared to full VMs.

Setting Up Jenkins on AWS (EC2 Instance)

We will set up Jenkins on Ubuntu AMI with Docker integration.

Step 1: Launch an EC2 Instance

• Use Ubuntu 22.04 AMI.

• Instance type: t2.medium (at least 4GB RAM for Jenkins).

• Open ports: 22 (SSH), 8080 (Jenkins), 2375 (Docker Daemon if needed).

Once the instance is launched, click on the launch instance and select the security group. Click on edit inbound rules and add the port range 8080 for Jenkins.

Step 2: ssh into EC2 through your terminal

ssh -i <path to key-pair> ubuntu@ip-aadress

Step 3: Install Jenkins on EC2

sudo apt update && sudo apt upgrade -y
sudo apt install -y openjdk-17-jdk wget
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/" | sudo tee /etc/apt/sources.list.d/jenkins.list
sudo apt update
sudo apt install -y jenkins
sudo systemctl enable --now jenkins

Jenkins should now be running on port 8080.

Go to your terminal and paste the command below which is the adminstrator password you would use to login to Jenkins and register.

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Click on install suggested plugins, next fill in the details required and follow the prompt

Step 4: Go back to your terminal and Install Docker on EC2

sudo apt install -y docker.io
sudo systemctl enable --now docker

✅ Docker installed successfully.

Step 5: Add Jenkins & Ubuntu User to Docker Group

sudo usermod -aG docker jenkins
sudo usermod -aG docker ubuntu
sudo systemctl restart jenkins

To verify run the following command

sudo su - jenkins

Once you are inside jenkins user, run a simple docker command of hello-world

docker run hello-world

Step 6: Install Docker Plugin in Jenkins Dashboard

• Go to Manage Jenkins > Plugin Manager.

  

• Search for "Docker Pipeline Plugin", then click on install.

  

• Restart Jenkins.

  

  First Jenkins Job: Running a Pipeline with Docker Agent

  Now that Jenkins and Docker are set up, let’s run a simple pipeline:

  Step 1: Create a New Pipeline Job

  1. Go to Jenkins Dashboard → New Item → Pipeline.

     

  2. Name it "Docker Test Pipeline", click on pipeline then ok and next.

     

  3. Scroll down to Pipeline Script and add the following:

pipeline {
    agent {
        docker { image 'node:latest' }  // Runs inside a Node.js container
    }
    stages {
        stage('Test') {
            steps {
                sh 'node --version'  // Check if Node.js is installed
            }
        }
    }
}

Step 2: Save & Run

• Click Build Now and check the logs.

  

• Click on the console output, you should see the Node.js version printed, meaning the pipeline ran successfully inside a Docker container!

  

  

  Conclusion

  In this first part of our series of streamlining CI/CD on AWS: Mastering Jenkins and Docker Agents (Part 1), we've successfully set up Jenkins on an EC2 instance and integrated Docker to streamline our build processes. By leveraging Docker agents, we have optimized resource usage and reduced costs, setting a strong foundation for efficient CI/CD operations. We also created a simple pipeline to demonstrate the seamless execution of tasks within Docker containers.

  In the upcoming parts, we will enhance our pipeline by integrating GitHub for code management, deploying a real-world application, and implementing webhooks for automated builds, further solidifying our CI/CD strategy. Stay tuned for more advanced configurations and practical applications!