Project Overview

This project demonstrates a complete CI/CD pipeline for a Bank Application using AWS DevOps tools. The pipeline includes:

Jenkins for CI/CD orchestration
SonarQube for code quality analysis
Nexus for artifact repository management
Docker for containerization
Trivy for security scanning
Kubernetes (EKS) for deployment

1. Jenkins Server Setup

Step 1: Launch EC2 Instance

Launch an Ubuntu EC2 instance with t2.large and 25GB storage for the Jenkins Server
Security group should allow inbound traffic on port 8080

Step 2: Install Jenkins

Connect to the instance using its public IP:

ssh -i <.pem key name> ubuntu@<EC2 public IP>

2. Update the server
sudo apt update

3. Install Java on the Jenkins Server

sudo apt install openjdk-17-jre-headless -y

4. Install Jenkins on the server:

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null

sudo apt-get update
sudo apt-get install jenkins -y

5. Verify the Jenkins status:

sudo systemctl status jenkins

6. Access Jenkins UI using the server IP address at port 8080:

<EC2 Public IP>:8080

7. Get the admin password:

/var/lib/jenkins/secrets/initialAdminPassword
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

8. Complete the setup by:

Installing suggested plugins
Creating an admin user
Configuring the Jenkins URL

Create Admin User with below details and click on save next

Now we will be able to view the Jenkins Dashboard

2. Plugin Setup

Install the following Jenkins plugins:

Blue Ocean
Pipeline: Stage View Version
SonarQube Scanner
Config File Provider
Maven Integration
Pipeline Maven Integration
Kubernetes Client API
Kubernetes Credentials
Kubernetes
Kubernetes CLI
Kubernetes Credentials Provider
Kubernetes :: Pipeline :: DevOps Steps
Docker
Docker Pipeline
Pipeline Stage View

3. Install Docker on Jenkins Server

# Add Docker's official GPG key
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update

# Install Docker packages
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

# Verify Docker installation
docker --version

# Set permissions for Docker socket
sudo chmod 666 /var/run/docker.sock

4. Install Trivy on Jenkins Server

sudo apt-get install wget gnupg
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb generic main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y

# Verify Trivy installation
trivy --version

5. SonarQube and Nexus Setup

Launch 2 EC2 instances (t2.medium with 20GB storage):

1 for SonarQube
1 for Nexus

For both servers:

Connect and update:

→ Connect to both servers using their public IP address and update using the below command

sudo apt update

→ Install Docker on Both Servers and Change the permission for the docker

sudo apt install docker.io -y
sudo chmod 666 /var/run/docker.sock

5.1 Nexus Setup

Install Nexus as a Docker container:
sudo docker run -d --name Nexus3 -p 8081:8081 sonatype/nexus3

Now we will able to access Nexus Repository using Nexus-Server public IP at port 8081

Click on sign in to login to Nexus

Get the admin password:

Get into container

docker ps
docker exec -it <container ID> /bin/bash
cd sonatype/nexus3/
cat admin.password

Complete the setup:

Login with the password
Set a new admin password
Disable anonymous access

setup new admin password

click on Next, and select “Disable anonymous access”

Click on Next and select Finish

Nexus Setup is completed.

5.2 SonarQube Setup

Install SonarQube as a Docker container:

sudo docker run -d --name Sonardevops -p 9000:9000 sonarqube:lts-community

Access SonarQube at <Sonar-Server-IP>:9000

Username: admin
Password: admin

Change the default password when prompted.

User Name: admin

Password: admin

Click on Login and update the password

sonarqube setup is completed successfully.

6. Configure Tools on Jenkins Server

6.1 SonarQube Scanner Configuration

Go to Dashboard → Manage Jenkins → Tools:

Add a SonarQube Scanner installation named "sonar-scanner"
Install automatically from Maven Central

6.2 Maven Configuration

Add Maven installation:

Name: "maven3"
Install automatically from Apache

6.3 Docker Configuration

Add Docker installation:

Name: "docker"
Install automatically

Click on Apply and Save.

7. Configure SonarQube Server on Jenkins

7.1 Generate Token in SonarQube

Go to SonarQube Server → Administration → Security → Users
Click on Tokens and Update Token
Create a token with a name and generate
Copy the generated token

7.2 Add Token to Jenkins Credentials

In Jenkins, go to Dashboard → Manage Jenkins → Credentials → Add Secret Text:

ID: sonar-token
Description: SonarQube Token
Secret: <Paste the token>

You will this page once you click on create

7.3 Add Docker Hub Credentials

Add Docker Hub credentials in Jenkins:

ID: docker-cred
Description: Docker Hub Credentials
Username: <Your Docker Hub username>
Password: <Your Docker Hub password>

7.4 Configure SonarQube in Jenkins

Go to Dashboard → Manage Jenkins → System:

Under SonarQube servers:
- Select SonarQube installations
- Click on Add SonarQube
- Name: sonar
- URL: <SonarQube server URL>
- Select the sonar token credential

7.5 Add SonarQube Webhook

In SonarQube:

Go to Administration → Configuration → Webhooks
Click on Create
Add URL: http://<jenkins-public-ip>:8080/sonarqube-webhook/

Click on Create

Modify the POM.xml

8. Configure Nexus Repository

8.1 Modify pom.xml

Update maven-releases and maven-snapshots URLs in pom.xml under distributionManagement:

<distributionManagement>
<repository>
<id>maven-releases</id>
<url>http://<nexus-ip>:8081/repository/maven-releases/</url>
</repository>
<snapshotRepository>
<id>maven-snapshots</id>
<url>http://<nexus-ip>:8081/repository/maven-snapshots/</url>
</snapshotRepository>
</distributionManagement>

Before:

Copy maven-releases and maven-snapshots URLs from Nexus Repository

Goto Server administration and configuration → Repository → Repositories

Copy maven-releses URL

Update in pom.xml file

Similary copy maven-snapshots URL and update in pom.xml

After:

8.2 Provide Credentials for Nexus

In Jenkins, go to Dashboard → Manage Jenkins → Managed files:

Add a new Config of type Global Maven settings.xml
ID: settings-maven
Modify settings.xml to include server details:

Select Type as: Global Maven settings.xml

ID: Enter Any Name [here settings-maven]

Click on Next, It will provide the settings.xml file

Modify the Server details in settings.xml file

Before:

After:

<servers>
<server>
<id>maven-releases</id>
<username>admin</username>
<password>your_nexus_password</password>
</server>
<server>
<id>maven-snapshots</id>
<username>admin</username>
<password>your_nexus_password</password>
</server>
</servers>

Added maven-releases and maven-snapshots server details with Nexus repos user ID and Password to access Nexus and push the artifacts to the Nexus repository

Click on Submit, settings file is added successfully

9. Create Jenkins Pipeline

Go to Jenkins Dashboard
Click on New Item, enter a name and select Pipeline
Under General, select "Discard old builds" and set Max # of builds to keep: 2
Add the pipeline script:

click on Ok

Under General, select Discard old builds and

set Max # of builds to keep: 2

Start writing the pipeline

Jenkins Pipeline:

pipeline {

Now we will be able to access the application Using Jenkins Server IP on port 5000.

Note: Use "adijaiswal/bankapp

" Docker image to ensure proper functionality.

Deploy Using Kubernetes

10.1 Server Setup for EKS

Launch an EC2 instance (t2.medium with 20GB storage)
Install AWS CLI:
bash
Copy

Connect to server and Update

Install AWS CLI on the Server

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version

Create an AWS Credential, ACCESS KEY and SECRET ACCESS KEY to access AWS and create EKS cluster

Configure AWS with your access keys:

aws configure

Install Terraform

sudo snap install terraform --classic

Clone the EKS Terraform repository:

git clone https://github.com/info4devops/EKS-Terraform.git

The cluster is created successfully

10.2 Configure RBAC for Kubernetes

mkdir RBAC

cd RBAC

Install kubectl:

sudo snap install kubectl --classic

Update kubeconfig file:

aws eks --region ap-south-1 update-kubeconfig --name devopssubroto-cluster

Create namespace:

kubectl create namespace webapps

Create service account, role, and role binding: svc.yml:

apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: webapps

role.yml:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: app-role
namespace: webapps
rules:
- apiGroups:
- ""
- apps
- autoscaling
- batch
- extensions
- policy
- rbac.authorization.k8s.io
resources:
- pods
- componentstatuses
- configmaps
- daemonsets
- deployments
- events
- endpoints
- horizontalpodautoscalers
- ingress
- jobs
- limitranges
- namespaces
- nodes
- secrets
- pods
- persistentvolumes
- persistentvolumeclaims
- resourcequotas
- replicasets
- replicationcontrollers
- serviceaccounts
- services
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Bind.yml:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: app-rolebinding
namespace: webapps
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: app-role
subjects:
- namespace: webapps
kind: ServiceAccount
name: jenkins

Create a secret for the service account token: secret.yml:

apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
name: mysecretname
annotations:
kubernetes.io/service-account.name: jenkins

Apply the secret:

kubectl apply -f secret.yml -n webapps

Get the token:

kubectl describe secret mysecretname -n webapps

Add the token to Jenkins credentials as "k8s-token"

10.3 Create Kubernetes Deployment Files

Create ds.yml:

---
# MySQL Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
spec:
selector:
matchLabels:
app: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: mysql
spec:
containers:
- image: mysql:8
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "Test@123"
- name: MYSQL_DATABASE
value: "bankappdb"
ports:
- containerPort: 3306
name: mysql
---
# MySQL Service
apiVersion: v1
kind: Service
metadata:
name: mysql-service
spec:
ports:
- port: 3306
selector:
app: mysql
---
# Java Application Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: bankapp
spec:
replicas: 1
selector:
matchLabels:
app: bankapp
template:
metadata:
labels:
app: bankapp
spec:
containers:
- name: bankapp
image: username/bankapp:latest
ports:
- containerPort: 8080
env:
- name: SPRING_DATASOURCE_URL
value: jdbc:mysql://mysql-service:3306/bankappdb?useSSL=false&serverTimezone=UTC&allowPublicKeyRetrieval=true
- name: SPRING_DATASOURCE_USERNAME
value: root
- name: SPRING_DATASOURCE_PASSWORD
value: Test@123
---
# Bank Application Service
apiVersion: v1
kind: Service
metadata:
name: bankapp-service
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 8080
selector:
app: bankapp

10.4 Install kubectl on Jenkins Server

Add these stages to your Jenkins pipeline:

pipeline {
agent any

tools{
maven 'maven3'
}

environment {
SCANNER_HOME= tool 'sonar-scanner'
}

stages {
stage('Git Checkout') {
steps {
git branch: 'main', url: 'https://github.com/subrotosharma/Bank-Application-Multi-Tier-CICD-Project.git'
}
}

stage('Compile') {
steps {
sh 'mvn compile'
}
}

stage('Test') {
steps {
sh 'mvn test -DskipTests=true'
}
}

stage('Trivy FS Scan') {
steps {
sh 'trivy fs --format table -o fs-report.html .'
}
}

stage('SonarQube Analysis') {
steps {
withSonarQubeEnv('sonar') {
sh "$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Bank-App -Dsonar.projectKey=Bank-App -Dsonar.java.binaries=target"
}
}
}

stage('Build') {
steps {
sh 'mvn package -DskipTests=true'
}
}

stage('Publish To Nexus') {
steps {
withMaven(globalMavenSettingsConfig: 'settings-maven', jdk: '', maven: 'maven3', mavenSettingsConfig: '', traceability: true) {
sh "mvn deploy -DskipTests=true"
}
}
}

stage('Docker Build Image') {
steps {
script {
withDockerRegistry(credentialsId: 'docker-cred', toolName: 'docker') {
sh " docker build -t username/bankapp:latest ."
}
}
}
}

stage('Trivy Image Scan') {
steps {
sh 'trivy image --format table -o fs-report.html username/bankapp:latest'
}
}

stage('Docker Push Image') {
steps {
script {
withDockerRegistry(credentialsId: 'docker-cred', toolName: 'docker') {
sh " docker push username/bankapp:latest"
}
}
}
}

stage('Deploy to K8s') {
steps {
script {
withKubeCredentials(kubectlCredentials: [[caCertificate: '', clusterName: 'devopssubroto-cluster', contextName: '', credentialsId: 'k8s-token', namespace: 'webapps', serverUrl: 'Cluserter-URL']]) {
sh "kubectl apply -f ds.yml -n webapps"
sleep 30
}
}
}
}

stage('Verify Deployment') {
steps {
script {
withKubeCredentials(kubectlCredentials: [[caCertificate: '', clusterName: 'devopssubroto-cluster', contextName: '', credentialsId: 'k8s-token', namespace: 'webapps', serverUrl: 'Cluster-URL']]) {
sh "kubectl get pods -n webapps"
sh "kubectl get svc -n webapps"
}
}
}
}

}
}

Now we will be able to access application using load balancer URL

Final Application:

11. Cleanup Resources

To destroy the EKS cluster when you're done:

terraform destroy --auto-approve

Final Application

After successful deployment, you can access the application using the Load Balancer URL provided by AWS.

This document provides a comprehensive guide to setting up a complete CI/CD pipeline for a Bank Application using various DevOps tools. It covers everything from Jenkins setup to deployment on Kubernetes.

Enterprise-Grade CI/CD Pipeline for Java Bank App Using Jenkins and AWS DevOps

Table of contents