Modern application delivery requires a seamless CI/CD pipeline to ensure code moves from development to production with minimal manual intervention. In this blog, I'll walk you through a DevOps project I recently worked on, where we implemented a multi-branch Jenkins pipeline integrated with Docker and Kubernetes for deploying an application on an Amazon EKS cluster. Let’s dive into the process.

This blog is based on the microservices deployment project available on my GitHub: https://github.com/CrystallyRains/11-Microservices.The repository contains the complete codebase, YAML configurations, and the Jenkinsfile required for this tutorial

Project Overview

The project involves:

Setting up Jenkins and Docker on a system.
Creating a multi-branch Jenkins pipeline.
Automating CI/CD with GitHub webhooks and Jenkins.
Building Docker images, pushing them to DockerHub.
Deploying the application to an EKS cluster using Kubernetes manifests.

Prerequisites

Before we begin, ensure you have the following installed on your system:

Java: Required for Jenkins.
Jenkins: Acts as the CI/CD tool.
Docker: To build and manage container images.

Additionally, ensure you have:

A DockerHub account (to store Docker images).
An AWS account with EKS setup permissions.

Step 1: Install Necessary Tools and Configure Jenkins

Install Jenkins:
- Download and install Jenkins.
- Start Jenkins and log in to the dashboard.
Install Docker:
- Install Docker and ensure the Docker daemon is running.
Integrate Jenkins with Docker:
- Add Jenkins user to the Docker group:
```
  sudo usermod -aG docker jenkins
```
  Restart Jenkins for changes to take effect.
Install Required Jenkins Plugins:
- Go to Manage Jenkins > Manage Plugins and install:
  - Docker Pipeline Plugin
  - Kubernetes Plugin
  - GitHub Plugin
  - Multibranch Pipeline Plugin
Add DockerHub Credentials in Jenkins:
- Navigate to Manage Jenkins > Manage Credentials.
- Add DockerHub credentials with:
  - ID: dockerhub-creds
  - Username: Your DockerHub username.
  - Password: Your DockerHub password.

Step 2: Set Up Multi-Branch Pipeline

Create a Multi-Branch Pipeline Job:
- In Jenkins, click New Item and select Multibranch Pipeline.
- Set the GitHub repository URL for the project.
Configure GitHub Webhook:
- In your GitHub repository:
  - Go to Settings > Webhooks.
  - Add a webhook with:
    - Payload URL: http://<jenkins-server-ip>:8080/github-webhook/
    - Content Type: application/json.
    - Trigger: Push events.

Add a Jenkinsfile to Each Branch:

Each branch in your GitHub repository should have its own Jenkinsfile. Here’s a sample CI pipeline:

  pipeline {
      agent any
      stages {
          stage('Checkout Code') {
              steps {
                  checkout scm
              }
          }
          stage('Build Docker Image') {
              steps {
                  script {
                      def appImage = docker.build("yourdockerhubusername/app:${env.BRANCH_NAME}-${env.BUILD_NUMBER}")
                  }
              }
          }
          stage('Push Image to DockerHub') {
              steps {
                  withDockerRegistry([credentialsId: 'dockerhub-creds', url: '']) {
                      script {
                          docker.image("yourdockerhubusername/app:${env.BRANCH_NAME}-${env.BUILD_NUMBER}").push()
                      }
                  }
              }
          }
      }
  }

Step 3: Deploy to Amazon EKS

After the CI pipeline builds and pushes the image to DockerHub, the next step is to deploy it to an Amazon EKS cluster.

Set Up Kubernetes Resources

Create Kubernetes Service Account:
- Service accounts allow Jenkins to interact with the Kubernetes cluster.

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: jenkins-sa
      namespace: default

Create Role and RoleBinding:

 apiVersion: rbac.authorization.k8s.io/v1
 kind: Role
 metadata:
   namespace: default
   name: jenkins-role
 rules:
   - apiGroups: [""]
     resources: ["pods", "services", "deployments"]
     verbs: ["get", "watch", "list", "create", "delete"]
 ---
 kind: RoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   name: jenkins-rolebinding
   namespace: default
 subjects:
   - kind: ServiceAccount
     name: jenkins-sa
     namespace: default
 roleRef:
   kind: Role
   name: jenkins-role
   apiGroup: rbac.authorization.k8s.io

Generate a Secret Token: Deploy the following my-secret.yml file:

 apiVersion: v1
 kind: Secret
 metadata:
   name: jenkins-token
   namespace: default
 type: Opaque
 data:
   token: <base64-encoded-token>

Deployment Jenkinsfile for Main Branch

Here’s an example Jenkinsfile for the CD pipeline:

pipeline {
    agent any
    environment {
        DOCKER_IMAGE = "yourdockerhubusername/app:main-${env.BUILD_NUMBER}"
    }
    stages {
        stage('Deploy to EKS') {
            steps {
                script {
                    sh '''
                    kubectl apply -f deployment.yml
                    kubectl apply -f service.yml
                    '''
                }
            }
        }
    }
}

Step 4: Kubernetes Deployment Files

deployment.yml:

 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: app-deployment
   namespace: default
 spec:
   replicas: 3
   selector:
     matchLabels:
       app: app
   template:
     metadata:
       labels:
         app: app
     spec:
       containers:
       - name: app
         image: yourdockerhubusername/app:main-latest
         ports:
         - containerPort: 80

service.yml:

 apiVersion: v1
 kind: Service
 metadata:
   name: app-service
   namespace: default
 spec:
   selector:
     app: app
   ports:
     - protocol: TCP
       port: 80
       targetPort: 80
   type: LoadBalancer

Final Workflow

Code is pushed to a branch in GitHub.
The GitHub webhook triggers Jenkins.
Jenkins builds and pushes the Docker image.
For the main branch, Jenkins deploys the application to EKS automatically.

Conclusion

This project showcases the power of automation in DevOps, integrating CI/CD pipelines with cloud-native tools like Docker and Kubernetes. With Jenkins orchestrating the workflow, we achieved a fully automated deployment pipeline to Amazon EKS, ensuring faster and more reliable releases.

Building a Complete CI/CD Pipeline for EKS Deployment with Jenkins and Docker

Table of contents