What is CI/CD? How It Works

Continuous Integration and Continuous Deployment (CI/CD) is a modern software development practice that automates the process of integrating code changes, testing, and deploying applications. It aims to shorten the development cycle, improve software quality, and enable teams to deliver updates more frequently and reliably.

Continuous Integration (CI):

CI is the practice of frequently integrating code changes into a shared repository. Developers commit their changes multiple times a day. Each integration triggers an automated build and testing process to ensure that the new code does not break the existing application.

Continuous Deployment (CD):

CD extends CI by automatically deploying all validated changes to a production environment. After passing automated tests, the code is automatically released, making new features and bug fixes available to users without manual intervention.

How CI/CD Works:

Code Commit: Developers push their changes to a version control system like Git.
Build: The CI/CD pipeline automatically compiles the code.
Testing: Automated tests are executed to check for bugs and ensure code quality.
Deployment: If tests pass, the changes are deployed to production or staging environments.
Monitoring: The system monitors the application in production to detect issues early.

Why Jenkins? Benefits for the Industry

Jenkins is one of the most popular open-source automation tools for implementing CI/CD. It enables developers to automate various stages of their software delivery pipeline.

Benefits of Jenkins:

Automation: Reduces manual work by automating code integration, testing, and deployment.
Open-Source: Free to use and has a large community for support.
Extensibility: Offers over 1,800 plugins to customize and extend functionalities.
Integration: Works seamlessly with tools like Git, Docker, Kubernetes, and various testing frameworks.
Efficiency: Automates repetitive tasks, enabling teams to focus on innovation and development.

Industry Benefits:

Faster Delivery: Reduces the time required to deploy new features and bug fixes.
Improved Quality: Automated testing ensures that issues are identified early.
Cost Efficiency: Reduces deployment errors and the need for manual testing, saving time and money.
Scalability: Supports large and complex projects with distributed build capabilities.

Impact on Work Culture Compared to Traditional Methods

Traditional Development Workflow:

Manual Processes: Code integration, testing, and deployment were done manually, increasing the risk of errors and delays.
Infrequent Releases: Software updates were released quarterly or annually, causing long development cycles.
Silos: Developers, testers, and operations teams worked in isolation, leading to communication gaps and slower issue resolution.

Modern CI/CD Workflow with Jenkins:

Collaboration: Developers, testers, and operations teams work together, fostering a DevOps culture.
Agility: Teams can respond quickly to market changes and user feedback.
Continuous Improvement: Automated pipelines allow for frequent releases, encouraging experimentation and innovation.
Reduced Stress: Automation minimizes human error, reducing deployment failures and enabling a more predictable development process.

Step-by-Step Guide to Setting Up Jenkins on AWS EC2

Setting up Jenkins on an AWS EC2 instance is a fundamental step toward automating your CI/CD pipeline. This guide will walk you through the entire process using an Ubuntu image, although the steps can be adapted for other Linux distributions.

1. Create an AWS EC2 Instance

Sign in to your AWS Management Console.
Launch a new EC2 instance.
Select Ubuntu Server 22.04 LTS (HVM), SSD Volume Type as the Amazon Machine Image (AMI).
Choose an instance type (e.g., t2.micro for free-tier eligibility).
Configure Security Group:
- Add an Inbound Rule to allow traffic on port 8080 (Jenkins) from 0.0.0.0/0 (IPv4).
- Optionally allow SSH (port 22) access from your IP address.
Launch the instance and download your private key (.pem) file.

2. Connect to the EC2 Instance

Open your terminal and navigate to the directory containing your .pem file.
Run the following command to connect to your instance:
```
  ssh -i <your-key.pem> ubuntu@<EC2-Public-IP>
```

3. Update the System Packages

For Ubuntu:

sudo apt update
sudo apt upgrade -y

For CentOS:

sudo yum update -y

4. Install Java

Jenkins requires Java to run. Install OpenJDK 17 on Ubuntu with the following command:

sudo apt install fontconfig openjdk-17-jre -y

5. Install Jenkins

Follow these steps to install Jenkins on Ubuntu:

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/" | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

sudo apt update
sudo apt install jenkins -y

For other Linux distributions, refer to the official Jenkins documentation.

6. Configure Security Group

If you did not add port 8080 during the EC2 creation process, follow these steps:

Go to EC2 Dashboard > Instances.
Select your instance and navigate to Security > Security Groups.
Edit the Inbound Rules and add:
- Type: Custom TCP
- Port Range: 8080
- Source: 0.0.0.0/0 (IPv4)

7. Access Jenkins

Open your browser and navigate to:
```
  http://<EC2-Public-IP>:8080
```

8. Retrieve Administrator Password

In your terminal (EC2 instance), run the following command:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the password and paste it into the Jenkins setup page in your browser.

9. Install Suggested Plugins

Click Install Suggested Plugins on the Jenkins setup page. Alternatively, you can manually select specific plugins.

10. Create Admin User

Follow the instructions to create an administrator username and password.
Complete the setup and start using Jenkins!

Creating a Jenkins Pipeline Job

Setting up a job in Jenkins is a crucial step toward automating your software development process. This guide will walk you through creating a Jenkins Pipeline job from scratch, along with enhancing your output visualization by using the Stage View Plugin.

Installing the Stage View Plugin (Optional but Recommended)

To get a better visualization of your pipeline stages, it is recommended to install the Stage View Plugin:

Navigate to Manage Jenkins from the Jenkins dashboard.
Click on Plugins.
Go to the Available Plugins tab.
Search for Stage View Plugin.
Install the plugin and choose Restart Jenkins after installation (recommended).

Restarting Jenkins ensures that the plugin is fully configured and operational.

Steps to Create a Jenkins Pipeline Job

1. Create a New Item

Click on New Item from the Jenkins dashboard.
Enter a name for your job (e.g., My First Pipeline Job).

2. Choose Job Type

Select Pipeline as the project type.
Click OK.

Why Pipeline is Recommended Over Freestyle Project:

Better Visualization: Pipelines allow for visual representation of stages using the Stage View Plugin.
Complex Workflows: Pipelines support multi-stage workflows and are more suitable for CI/CD processes.
Code as Configuration: The pipeline is defined using a script, making it version-controlled and easily reproducible.

3. Configure the Pipeline Job

After creating the job, you will be directed to the job configuration page.
Optionally, add a description of your project in the Description field.

4. Define the Pipeline Script

Scroll down to the Pipeline section.
Under Definition, select Pipeline script.
Paste the following pipeline script in the script box:

pipeline {
    agent any

    stages {
        stage('Create A Folder') {
            steps {
                sh 'mkdir -p first-job'
            }
        }
        stage('Create A File Under The Folder') {
            steps {
                sh 'cd first-job && touch job.txt'
            }
        }
        stage('Put Some Data in the File') {
            steps {
                sh 'echo "This is my first Jenkins Pipeline" > first-job/job.txt'
            }
        }
    }
}

5. Save and Build the Job

Click Save at the bottom of the page.
Click Build Now on the left sidebar.

6. View the Pipeline Execution

After the build is complete, click on the build number.
Click on Pipeline or Stage View to visualize the stages and monitor the progress.

After the build, you can see this in your terminal if you navigate to that location.

Additional Resources

This is the GitHub repository containing all the scripts used in this blog: https://github.com/Debjyoti2004/jenkins-in-action

Creating a Job with Parameters

Parameterized jobs in Jenkins allow you to pass values dynamically when triggering a build.

1. Enable Parameters for a Job

Open your Pipeline job configuration.
Check This project is parameterized.
Click Add Parameter and choose the type (e.g., String Parameter).
Provide a name (e.g., FOLDER_NAME) and default value (e.g., my-folder).

2. Modify the Pipeline Script to Use Parameters

pipeline {
    agent any

    stages {
        stage('Create A Folder') {
            steps {
                sh 'mkdir -p first-job'
            }
        }
        stage('Create A File Under The Folder') {
            steps {
                sh 'cd first-job && touch job.txt'
            }
        }
        stage('Put some data in the file') {
            steps {
                sh "echo ${FirstParameter} > first-job/job.txt"
            }
        }
    }
}

3. Save and Build with Parameters

Click Build with Parameters and enter the desired folder name.
Monitor the build and verify the folder is created with the provided name.

Scheduling Jobs (Cron Jobs) in Jenkins🕵

Jenkins allows you to schedule jobs using a cron-like syntax.

1. Configure Build Triggers

Open the job configuration.
In the Build Triggers section, check Build periodically.
Enter a cron expression (e.g., H * * * * to run every hour).

2. Useful Cron Example

Expression	Meaning
`H * * * *`	Every hour
`H/15 * * * *`	Every 15 minutes
`H 0 * * *`	Every day at midnight

3. Cron Expression Tester

Use the Cron Tester Tool to validate your cron expressions before adding them to Jenkins.

How to Clone Code in Jenkins and Run an Express Application

Cloning Code from GitHub in Jenkins

Step 1: Create a New Job

Open Jenkins Dashboard.
Click on New Item.
Provide a name for your job.
Select Pipeline and click OK.

Step 2: Add Pipeline Script

Copy and paste the following pipeline script to clone code from GitHub:

pipeline {
    agent any

    stages {
        stage('Clone Code From GitHub') {
            steps {
                cleanWs()
                script {
                    try {
                        git branch: 'master', url: 'https://github.com/Debjyoti2004/Nginx_Simple_Project.git'
                    } catch (Exception e) {
                        error "Failed to clone repository: ${e.getMessage()}"
                    }
                }
            }
        }
    }
}

Step 3: Save and Build

Click Save.
Click Build Now.
Check the console output for success.

Cloning and Running an Express Application in Jenkins🚀

Step 1: Install Node.js, npm, and PM2 in Your EC2 Instance

If you are using an Ubuntu EC2 instance, run the following commands to install Node.js, npm, and PM2:

sudo apt update
sudo apt install nodejs  # Install Nodejs
node -v

sudo apt install npm   # Install npm
npm --version

sudo npm install -g pm2   # Install Pm2
pm2 --version

You can also follow this DigitalOcean Guide for detailed steps.

Step 2: Create a New Job in Jenkins

Follow the steps mentioned above to create a new Pipeline job.

Step 3: Open Port 3000 in Security Group

Ensure you open port 3000 in your EC2 instance security group inbound rules.

Step 4: Add the Pipeline Script

Use the following pipeline script to clone and run an Express application:

pipeline {
    agent any

    stages {
        stage('Verify Setup') {
            steps {
                sh '''
                node -v
                npm -v
                '''
                echo 'Verifying Setup Successfully'
            }
        }

        stage('Clone Code From GitHub') {
            steps {
                cleanWs()
                git branch: 'master', url: 'https://github.com/Debjyoti2004/node-jenkins-deploy.git'
                echo 'Cloning code from GitHub Successfully'
            }
        }

        stage('Install Dependencies') {
            steps {
                sh 'npm install'
                echo 'Installing dependencies Successfully'
            }
        }

        stage('Run Application with PM2') {
            steps {
                echo 'Running application using PM2'
                sh '''
                npx pm2 delete my-node-app || true
                npx pm2 start index.js --name my-node-app
                npx pm2 save
                '''
                echo 'Application started and saved successfully in PM2'
            }
        }
    }
}

Why Use PM2?

PM2 is a process manager for Node.js applications that ensures your application stays running even if the server restarts.

Step 5: Save and Build

Click Save.
Click Build Now.
You will see the build stages and outputs in Jenkins.

Step 6: Access Your Application

In your browser, go to:

http://<Public-IP>:3000

You should see your application running.

If you find that script difficult, you can follow a simple script that I wrote and stored in this GitHub repo. Check it out here: GitHub Repository - Jenkins in Action

How to Create an Agent in Jenkins and Run an Express Application on It🚀

Introduction

In this blog, we will cover how to set up an agent (node) in Jenkins to run your jobs on a separate server. We will also run a simple Express application on the agent machine using Jenkins Pipeline.

Prerequisites

Two servers (e.g., EC2 instances on AWS)
- Main Server: Jenkins is installed here.
- Agent Server: The destination server where jobs will be executed.
Basic knowledge of Jenkins and Linux commands.

Step 1: Generate SSH Keys

SSH into your Main Server (Jenkins server).
Generate an SSH key pair:
```
 ssh-keygen
```
The keys are stored in ~/.ssh.
Navigate to the directory:
```
 cd ~/.ssh
 ls
```
Copy the content of the public key:
```
 cat id_rsa.pub
```

Step 2: Add Public Key to Agent Server

SSH into the Agent Server.
Navigate to ~/.ssh:
```
 cd ~/.ssh
 ls
```
Edit the authorized_keys file:
```
 vi authorized_keys
```
Press i to enter INSERT mode and paste the public key.
Press ESC, type :wq, and press Enter to save and exit.

Step 3: Add Agent Node in Jenkins

Go to Jenkins Dashboard -> Manage Jenkins -> Manage Nodes and Clouds -> New Node.
Enter a Node Name (e.g., dev) and choose Permanent Agent. Click OK.
Fill in the details:
- Remote Root Directory: /home/ubuntu (This is the working directory in the agent server where Jenkins will perform the tasks).
- Labels: dev (Labels help identify this machine in pipeline scripts, e.g., agent { label 'dev' }).
- Launch Method: Launch agent via SSH.
Under Host, enter the Public IP Address of the Agent Server.
If you don't have credentials:
- Click Add -> Jenkins -> Kind: SSH Username with Private Key.
- ID: Any name (e.g., agent-key).
- Description: Any description.
- Username: ubuntu (or centos, ec2-user depending on the image).
- Private Key: Enter directly -> Paste the private key from your Main Server (id_rsa content).
- Host Key Verification Strategy: Choose Non verifying Verification Strategy.
Click Save.
Check if the agent is connected (it will show a green icon).

Step 4: Run Express Application on Agent

Install Node.js, npm, and PM2 on Agent Server

SSH into your Agent Server and run:

sudo apt update
sudo apt install nodejs
node -v
sudo apt install npm
npm --version
sudo npm install -g pm2
pm2 --version

Add Port 3000 to Inbound Rules

Go to AWS Security Group Settings for the Agent Server and add Inbound Rule:

Type: Custom TCP
Port Range: 3000
Source: Anywhere (0.0.0.0/0)

Create Jenkins Job

Go to Jenkins Dashboard -> New Item -> Enter a name -> Pipeline -> OK.
In the Pipeline Definition section, paste this script:

pipeline {
    agent { label 'dev' }

    stages {
        stage('Verify Setup') {
            steps {
                sh '''
                node -v
                npm -v
                '''
                echo 'Verifying Setup Successfully'
            }
        }
        stage('Clone Code From GitHub') {
            steps {
                cleanWs()
                git branch: 'master', url: 'https://github.com/Debjyoti2004/node-jenkins-deploy.git'
                echo 'Cloning code from GitHub Successfully'
            }
        }
        stage('Install Dependencies') {
            steps {
                sh 'npm install'
                echo 'Installing dependencies Successfully'
            }
        }
        stage('Run Application with PM2') {
            steps {
                echo 'Running application using PM2'
                sh '''
                npx pm2 delete my-node-app || true
                npx pm2 start index.js --name my-node-app
                npx pm2 save
                '''
                echo 'Application started and saved successfully in PM2'
            }
        }
    }
}

Click Save and Build Now.
Once the job is successful, access your application in a browser:
```
 http://<Agent_Server_Public_IP>:3000
```

Why We Use Labels

Labels help Jenkins decide which node (agent) should execute a specific job. For example:

agent { label 'dev' }

This tells Jenkins to run the pipeline on the agent machine labeled dev.

Why Remote Root Directory

This is the directory on the agent server where Jenkins will execute jobs and store workspace files. Example: /home/ubuntu.

User Management in Jenkins: Setting Up Roles and Environment Variables🚀

Jenkins provides role-based user management, allowing you to assign roles with specific permissions to different users. This ensures better security and controlled access. In this blog, we’ll go step by step on how to:

Install Role-Based Authorization Strategy Plugin
Create a New User
Assign Roles to Users

1. Install Role-Based Authorization Strategy Plugin

Before we manage users and assign roles, we need to install the Role-Based Authorization Strategy Plugin.

Steps to Install the Plugin:

Open Jenkins Dashboard.
Click on Manage Jenkins → Manage Plugins.
Go to the Available tab and search for Role-Based Authorization Strategy.
Select it and click Install without Restart.

Now, wait for the installation to complete.

2. Create a New User in Jenkins

Once the plugin is installed, let's create a new user.

Steps to Create a User:

Go to Manage Jenkins → Manage Users.
Click on Create User.
Fill in the details:
- Username (e.g., alex)
- Password
- Confirm Password
- Full Name
- Email (optional)

Verify the User:

You can verify the newly created user by logging in with the username and password at your Jenkins URL:

http://<JENKINS_PUBLIC_IP>:8080

Try logging in with the new credentials to confirm the user was created successfully.

3. Assign a Role to the User

To restrict user access based on roles, we need to assign specific permissions.

Step 1: Enable Role-Based Authorization

Go to Manage Jenkins → Configure Global Security.
Scroll down to the Authorization section.
Select Role-Based Strategy and click Save.

Step 2: Create a Role

Go to Manage Jenkins → Manage and Assign Roles.
Click on Manage Roles.
In the Role to Add box, type a role name (e.g., developer) and click Add.
Now, select permissions for this role (e.g., build jobs, read-only access, etc.).
Click Save.

Step 3: Assign the Role to a User

Go back to Manage Jenkins → Manage and Assign Roles.
Click on Assign Roles.
Find the user you created (e.g., alex).
Assign the role by checking the box next to the role name.
Click Save.

✅ Now, when alex logs in, they will only have access based on the role permissions.

Creating Environment Variables in Jenkins👇🏻

Environment variables help store important values, like database URLs, API keys, and system paths. We can create global environment variables in Jenkins and use them in our pipeline scripts.

Steps to Add an Environment Variable:

Go to Manage Jenkins → Configure System.
Scroll down to Global Properties.
Check the box Environment Variables.
Click Add and enter:
- Name: MY_ENV_VAR
- Value: Hello Jenkins!
Click Save.

How to Use an Environment Variable in a Jenkins Pipeline

You can use environment variables inside a pipeline script like this:

pipeline {
    agent any

    environment {
        MY_ENV_VAR = 'Hello Jenkins!'  // Define inline OR use the global one
    }

    stages {
        stage('Print Environment Variable') {
            steps {
                echo "The Environment Variable is: $MY_ENV_VAR"
            }
        }
    }
}

When you build the job, Jenkins will display:

The Environment Variable is: Hello Jenkins!

Conclusion

I covered all the important things in Jenkins that I also learned during my own journey. Keep exploring and happy coding! 🚀

Mastering CI/CD: Hands-On Tasks to Deploy Express Apps with Jenkins! 🚀