Secure CI/CD Deployment for the Bingo Application using DevSecOps on AWS

Introduction

This guide provides step-by-step instructions for implementing a complete DevSecOps pipeline for the Bingo application. The project combines various technologies and tools to create a secure, containerized application with automated testing, security scanning, and deployment capabilities.

Technology Stack

• Infrastructure: AWS EC2, EKS, Route53

• CI/CD: Jenkins

• Containerization: Docker, Kubernetes

• Security Scanning: SonarQube, Trivy, OWASP Dependency Check

• Infrastructure as Code: Terraform

• Command Line Tools: kubectl, AWS CLI, eksctl

Project Overview

The project implements a full CI/CD pipeline for the Bingo application with the following features:

• Automated code quality scanning with SonarQube

• Vulnerability scanning with OWASP Dependency Check and Trivy

• Containerization with Docker

• Deployment to Kubernetes (EKS)

• DNS configuration for public access

Prerequisites

• AWS Account with appropriate permissions

• Basic knowledge of AWS services (EC2, EKS, IAM)

• Understanding of CI/CD concepts

• Familiarity with Docker and Kubernetes

Step 1: EC2 Instance Setup

1. Log in to your AWS console and navigate to EC2

2. Click on "Launch instance"

3. Configure the instance:

   • Name: bingo-server

   • OS: Ubuntu

   • Instance type: t2.medium (recommended for Jenkins)

   • Create a new key pair or select an existing one

   • Configure security group to allow these ports:

     • 8080 (Jenkins)

     • 3000 (Bingo application)

     • 9000 (SonarQube)

     • 80 (HTTP)

     • 443 (HTTPS)

4. Launch the instance and wait for initialization

Step 2: Tools Installation

Connect to your EC2 instance:

ssh -i /path/to/your-key.pem ubuntu@your-instance-public-ip

Install all required tools using the installation script below:

Necessary installation need to be bingo application server to run the application

Purpose: This guide automates the installation of essential DevOps tools in an Ubuntu environment (e.g., EC2 instance). These tools support CI/CD, containerization, security scanning, infrastructure provisioning, Kubernetes cluster management, and cloud CLI access.

🛠️ Tools Covered

1. Jenkins

2. Docker & SonarQube

3. Trivy

4. Terraform

5. kubectl

6. AWS CLI

7. Eksctl

1️⃣ Jenkins – Automate Your CI/CD Pipelines

Jenkins is a popular open-source automation server used to build, test, and deploy software. It allows continuous integration and delivery with numerous plugins.

📥 Installation

sudo apt update -y

sleep 5

sudo apt install fontconfig openjdk-17-jre -y

sleep 15

# Add Jenkins repo key and source list

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null

sudo apt-get update -y

sleep 5

sudo apt-get install jenkins -y

sleep 10

sudo systemctl enable jenkins

sudo systemctl start jenkins

2️⃣ Docker & SonarQube – Containerization and Code Quality

Docker allows applications to run in isolated containers, ensuring consistency across environments.
SonarQube is a platform for static code analysis and continuous inspection to detect bugs, code smells, and security vulnerabilities.

📥 Installation

sudo apt-get update -y

sudo apt-get install docker.io -y

sleep 5

# Grant docker access

sudo usermod -aG docker ubuntu

sudo usermod -aG docker jenkins

newgrp docker

sleep 5

sudo chmod 777 /var/run/docker.sock

sleep 10

# Run SonarQube in a container

docker run -d --name sonar -p 9000:9000 sonarqube:lts-community

3️ Trivy – Security Vulnerability Scanner

Trivy is a comprehensive and easy-to-use vulnerability scanner for containers and other artifacts.

📥 Installation

sudo apt-get install wget apt-transport-https gnupg lsb-release -y

sleep 5

wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null

echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list

sudo apt-get update -y

sleep 5

sudo apt-get install trivy -y

4️ Terraform – Infrastructure as Code

Terraform by HashiCorp is an IaC tool that allows you to provision, manage, and version infrastructure efficiently using declarative configuration files.

📥 Installation

sudo apt install wget -y

wget https://releases.hashicorp.com/terraform/1.10.5/terraform_1.10.5_linux_amd64.zip

sleep 5

sudo apt install unzip -y

unzip terraform_1.10.5_linux_amd64.zip

sudo mv terraform /usr/local/bin/

terraform -v

5️ kubectl – Kubernetes Command Line Tool

kubectl is the official CLI tool for interacting with Kubernetes clusters.

📥 Installation

sudo apt update

sleep 5

sudo apt install curl -y

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

sleep 5

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

kubectl version --client

6️ AWS CLI – Control AWS from the Command Line

AWS CLI is a unified tool to manage AWS services from the command line. It enables users to interact with AWS infrastructure programmatically.

📥 Installation

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

sleep 5

sudo apt-get install unzip -y

unzip awscliv2.zip

sudo ./aws/install

7️ eksctl – Manage EKS Clusters Easily

eksctl is a simple CLI tool for creating and managing Kubernetes clusters on AWS EKS.

📥 Installation

curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp

sleep 5

sudo mv /tmp/eksctl /usr/local/bin

# Verify installation

echo "Verifying installations..."

eksctl version

kubectl version --client

terraform -v

docker -v

aws --version

java --version

echo "All tools installed and verified."

Save this script in a file (e.g., install_tools.sh), make it executable (chmod +x install_tools.sh), and run it on your EC2 instance.

Step 3: Jenkins Configuration

1. Access Jenkins in your browser by navigating to: http://your-instance-public-ip:8080

2. Get the initial admin password from the EC2 instance:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

3. Install recommended plugins when prompted

4. Create an admin user when prompted

5. Install additional required plugins:

   • Navigate to: "Manage Jenkins" > "Manage Plugins" > "Available"

   • Search for and install the following plugins:

     • Eclipse Temurin installer

     • SonarQube Scanner

     • NodeJS

     • OWASP Dependency-Check

     • Pipeline: Stage View

     • Docker

     • Docker Commons

     • Docker Pipeline

     • Docker API

     • docker-build-step

Step 4: Configure Tools in Jenkins

1. Navigate to "Manage Jenkins" > "Tools"

2. Configure JDK:

   • Name: jdk-17

   • Installation method: Install from adoptium.net

   • Select JDK 17

3. Configure SonarQube Scanner:

   • Name: sonar-scanner

   • Install automatically: Yes

   • Install latest version

4. Configure NodeJS:

   • Name: node16

   • Install automatically: Yes

   • Install NodeJS 16.x

5. Configure Docker:

   • Name: docker

   • Installation method: Use system Docker installation

Step 5: Configure System Settings in Jenkins

1. Navigate to "Manage Jenkins" > "System" (or "Configure System")

2. Configure SonarQube servers:

   • Add SonarQube:

     • Name: sonar-server

     • Server URL: http://localhost:9000

     • Server authentication token: Add your SonarQube token (you'll need to generate this in SonarQube)

Step 6: Configure Docker Hub Credentials in Jenkins

1. Navigate to "Manage Jenkins" > "Credentials" > "System" > "Global credentials"

2. Add Docker Hub credentials:

   • Kind: Username with password

   • ID: docker

   • Username: your Docker Hub username

   • Password: your Docker Hub password

Step 7: Configure SonarQube

1. Access SonarQube at http://your-instance-public-ip:9000

2. Default login: admin/admin (change this after first login)

3. Go to "My Account" > "Security" and generate a token for Jenkins

4. Add this token to Jenkins as credential with ID sonar-token

Step 8: Set Up GitHub Webhook

1. Go to your GitHub repository (Fork or create https://github.com/subrotosharma/bingo1.git or use your own)

2. Navigate to Settings > Webhooks > Add webhook

3. Set the Payload URL to: http://your-instance-public-ip:8080/github-webhook/

4. Content type: application/json

5. Select "Just the push event"

6. Add the webhook

Step 9: Create Jenkins Pipeline

1. In Jenkins, click "New Item"

2. Enter name: "DevSecOps-Project-Bingo"

3. Select "Pipeline" and click OK

4. SonarQube Webhook configuration

5. Configure the pipeline:

   • Check "GitHub project" and add your repository URL

   • Under "Build Triggers", check "GitHub hook trigger for GITScm polling"

   • Under "Pipeline", select "Pipeline script" and paste the following pipeline script:

pipeline{

agent any

tools{

jdk 'jdk-17'

nodejs 'node16'

}

environment {

SCANNER_HOME=tool 'sonar-scanner'

}

stages {

stage('clean workspace'){

steps{

cleanWs()

}

}

stage('Checkout from Git'){

steps{

git branch: 'main', url: 'https://github.com/subrotosharma/bingo1.git'

}

}

stage("Sonarqube Analysis "){

steps{

withSonarQubeEnv('sonar-server') {

sh '''$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Bingo \

-Dsonar.projectKey=Bingo '''

}

}

}

stage("quality gate"){

steps {

script {

waitForQualityGate abortPipeline: false, credentialsId: 'sonar-token'

}

}

}

stage('Install Dependencies') {

steps {

sh "npm install"

}

}

stage('OWASP FS SCAN') {

steps {

dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP'

dependencyCheckPublisher pattern: '**/dependency-check-report.xml'

}

}

stage('TRIVY FS SCAN') {

steps {

sh "trivy fs . > trivyfs.txt"

}

}

stage("Docker Build & Push"){

steps{

script{

withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){

sh "docker build -t bingo ."

sh "docker tag bingo yourDockerHubUsername/bingo:latest"

sh "docker push yourDockerHubUsername/bingo:latest"

}

}

}

}

stage("TRIVY"){

steps{

sh "trivy image yourDockerHubUsername/bingo:latest > trivyimage.txt"

}

}

stage('Deploy to container'){

steps{

sh 'docker run -d --name bingo -p 3000:3000 yourDockerHubUsername/bingo:latest'

}

}

}

}

Note: Replace yourDockerHubUsername with your actual Docker Hub username.

5. Click "Save" to save the pipeline configuration

Step 10: Run the Pipeline

1. Click "Build Now" to manually trigger the pipeline, or push a change to the repository to trigger it via webhook

2. Monitor the pipeline progress and check each stage's logs for any issues

3. Once complete, you should be able to access the application at http://your-instance-public-ip:3000

Check docker image in docker registry and deploy to container

Sonarqube Scannner

Now we will check trivvy scan file. For that we should go to the specific build and clicked on it. Then we will see the workspace and then click on workspace and we will see the trivvy.txt file. Clicking on the trivvy file we will see the result

Check the application through browser

Step 11: Set Up EKS (Elastic Kubernetes Service)

Create an IAM Role

1. Go to AWS IAM console

2. Create a new role called bingouser with:

   • Use case: EC2

   • Add the following permissions:

     • IAMFullAccess

     • AmazonVPCFullAccess

     • AmazonEC2FullAccess

     • AmazonCloudFormationFullAccess

     • AdministratorAccess

Configure AWS CLI

1. Run aws configure and provide:

   • AWS Access Key ID

   • AWS Secret Access Key

   • Default region: us-east-1

   • Default output format: json

Create EKS Cluster

Run the following command to create your EKS cluster:

eksctl create cluster \

--name bingo-cluster \

--region us-east-1 \

--node-type t2.medium \

--nodes-min 2 \

--nodes-max 2 \

--zones us-east-1a,us-east-1b

This process will take 15-20 minutes to complete.

Step 12: Deploy Bingo App to EKS

1. Create a file named manifest.yaml with the following content:

---

apiVersion: apps/v1

kind: Deployment

metadata:

name: bingo-deployment

spec:

replicas: 2

strategy:

type: RollingUpdate

selector:

matchLabels:

app: bingo

template:

metadata:

labels:

app: bingo

spec:

containers:

- name: bingo-container

image: yourDockerHubUsername/bingo

ports:

- containerPort: 3000

---

apiVersion: v1

kind: Service

metadata:

name: bingo-service

spec:

type: LoadBalancer

selector:

app: bingo

ports:

- port: 80

targetPort: 3000

Note: Replace yourDockerHubUsername with your actual Docker Hub username.

2. Apply the manifest to your cluster:

kubectl apply -f manifest.yaml

3. Check deployment status:

kubectl get deployments

kubectl get pods

kubectl get services

4. Wait for the LoadBalancer to be provisioned and note the external IP/DNS

Step 13: Set Up DNS with Route53 (Optional)

1. Go to AWS Route53 console

2. Create a new record in your hosted zone:

   • Record name: Choose a subdomain (e.g., bingo.yourdomain.com)

   • Record type: A

   • Route traffic to: Alias to Application and Classic Load Balancer

   • Choose region: us-east-1

   • Choose load balancer: Select the EKS load balancer

3. Create the record and wait for DNS propagation (can take up to 48 hours)

Step 14: Access Your Application

1. Through EKS LoadBalancer: Use the external IP/DNS from kubectl get services

2. Through custom domain: If you set up Route53, use your subdomain (e.g., bingo.yourdomain.com)

Step 15: Verify Security Reports

To check the security reports generated during the pipeline:

1. SonarQube results: Access at http://your-instance-public-ip:9000

2. Trivy scan results: In Jenkins, navigate to the specific build > Workspace > trivyfs.txt and trivyimage.txt

Conclusion

Congratulations! You have successfully set up a complete DevSecOps pipeline for the Bingo application. This setup includes code quality scanning, vulnerability assessment, containerization, and deployment to Kubernetes with load balancing.

Troubleshooting Tips

1. Jenkins issues:

   • Check Jenkins logs:

sudo journalctl -u jenkins

• Ensure proper permissions:

sudo chown -R jenkins:jenkins /var/lib/jenkins/

2. Docker issues:

   • Verify Docker service:

sudo systemctl status docker

• Check user permissions: Ensure jenkins user is in docker group

3. EKS issues:

   • Verify cluster state:

eksctl get cluster

• Check node status:

kubectl get nodes

• Pod issues:

kubectl describe pod <pod-name>

4. Application access issues:

Check service: kubectl get svc bingo-service

• Check security groups: Ensure they allow traffic on port 80

5. DNS issues:

   • Verify record creation in Route53

   • Test DNS resolution: nslookup your-domain