Introduction

In this article, we will explore the deployment of a .NET-based application, a scenario frequently encountered by many organizations. Our deployment process will leverage Jenkins as the CI/CD tool and will involve deploying the application in Docker containers within a Kubernetes cluster.

This article aims to provide a detailed guide on the deployment process and includes comprehensive metrics, such as the CPU performance of the instance where the project is launched.

Step 1: Clone the GitHub Repository

Clone the repository containing the DotNet Application deployment scripts

git clone https://github.com/Saurabh-DevOpsVoyager77/DotNet-App-Deployment-DevSecOps.git

Step 2: Building an Infra using terraform

We'll build a simple infrastructure using Terraform. The infrastructure includes a user data script that installs Jenkins, Docker, and Trivy and starts a SonarQube container on port 9000. We'll run the Terraform commands to initialize, validate, plan, and apply the infrastructure configuration.

Run Terraform Commands:

cd Instance_Terraform/
terraform init
terraform validate
terraform plan
terraform apply --auto-approve

Using the script provided below, we will install Jenkins, Docker, Trivy, and other necessary packages. Additionally, the script will start the SonarQube container. Please note that if you have followed the previous steps, this script may have already been executed.

#!/bin/bash
sudo apt update -y
sudo touch /etc/apt/keyrings/adoptium.asc
sudo wget -O /etc/apt/keyrings/adoptium.asc https://packages.adoptium.net/artifactory/api/gpg/key/public
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | sudo tee /etc/apt/sources.list.d/adoptium.list
sudo apt update -y
sudo apt install temurin-17-jdk -y
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
                  /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
                  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
                              /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins

#Install docker
sudo apt install docker.io -y
sudo usermod -aG docker ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock
docker version

#SonarQube Container
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community

# Install Trivy
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y

Go to the AWS Dashboard in the EC2 service, where you can see the Instance.

Step 3: Set Up SonarQube and Jenkins

A. SonarQube

• Copy the public IP of your server/machine.

• Open your browser and navigate to <public-ip>:9000.

• When the SonarQube window opens, log in with the initial credentials:

Username: admin

Password: admin

• Update your password with the New Password.

• You will see the welcome window of SonarQube.

B. Jenkins

• Open your browser and navigate to <public-ip>:8080.

• Connect to your EC2 instance

• Run the following commands:

sudo su
cat /var/lib/jenkins/secrets/initialAdminPassword

• Copy the output (this is your Jenkins initial admin password).

• Paste the password into Jenkins to unlock it.

• Install the suggested plugins.

• Set up your Jenkins user.

Successfully you have entered into the Jenkins dashboard!

Step 4: CI/CD Pipeline

A. Installation of Plugins

Go to the Jenkins Dashboard -> Manage Jenkins -> Plugins and Install the Following Plugins:

• Eclipse Temurin Installer: Install without restart

• SonarQube Scanner: Install without restart

• OWASP Dependency-Check Plugin

• Download Docker-related Plugins: Docker, Docker Commons, Docker Pipeline, Docker API

• Kubernetes

• Kubernetes CLI

• Kubernetes Client API

• Kubernetes Pipeline DevOps Steps

B. Add Credentials for SonarQube and Docker

I. SonarQube Credentials Setup:

• Go to http://publicip:9000.

• Log in with your username and password.

• Click on Administration → Security → Users → Token → Generate Token.

• Set token_name as Sonar-token.

• Copy the Token.

II. Configure Jenkins:

• Copy the generated token.

• Go to your Jenkins dashboard.

• Navigate to Manage Jenkins → Credentials → System.

• Click on Global → Add Credentials.

• Select Secret text from the dropdown.

• Set the Secret as your token.

• Set the ID as jenkins.

• Click on Create.

III. Setup projects in SonarQube for Jenkins

• Go to your SonarQube server.

• Click on projects and in the name field type DotNet-Deployment-DevSecOps .

• Click on set up. Then you can see the screen like below

• Click on the above option i.e Locally.

• Click on Generate.

• Click on continue.

• SonarQube Project for Jenkins is setup now.

• Now, go to Dashboard → Manage Jenkins → System

IV. Setup Docker Credentials

• Go to your Jenkins dashboard.

• Navigate to Manage Jenkins → Manage Credentials.

• Click on Global → Add Credentials.

• Provide your DockerHub username and password.

• Set the ID as docker.

• Click on Create.

C. Setup Tools for Jenkins

I.Add JDK:

• Go to Manage Jenkins → Tools.

• Click on Add JDK and select the installer from adoptium.net.

• Choose JDK version 17.0.8.1+1 and name it as jdk17.

II. Add Docker:

• Click on Add Docker.

• Set the name as docker.

• Choose the installer option to download from docker.com.

III. Add SonarQube Scanner:

• Add Sonar Scanner.

• Name it as sonar-scanner/SonarQubeScanner.

IV. Add OWASP Dependency-Check:

• Add Dependency Check.

• Name it as DP-Check.

• Select the installer option to install from github.com.

• Click Apply & Save.

D. Configure Global Settings for SonarQube and Setup Webhooks

I. Configure Global Settings:

• Go to Manage Jenkins → System → Add SonarQube Servers.

• Set the name as sonar-server.

• Set the server URL as http://public_ip:9000.

• Set the server authentication token as jenkins(It is created in sonarqube security configuration).

II. Setup Webhooks in SonarQube:

• Go to Administration → Configuration → Webhooks.

• Create a webhook.

• Click on Create.

• Enter the URL as http://jenkins-public-ip:8080/sonarqube-webhook/.

E. Running the Pipeline

• Create a New Pipeline:

• Go to New Item → Select Pipeline.

• Name it as DotNet-pipeline

• Paste the below pipeline like this

pipeline {
    agent any

    tools {
        jdk 'jdk 17'
        // Assuming 'sonar-scanner' tool configuration is correctly set up in Jenkins Global Tool Configuration
    }

    environment {
        SCANNER_HOME = tool 'sonar-scanner'
    }

    stages {
        stage('Clean Workspace') {
            steps {
                cleanWs()
            }
        }

        stage('Checkout From Git') {
            steps {
                git branch: 'main', url: 'https://github.com/Saurabh-DevOpsVoyager77/DotNet-App-Deployment-DevSecOps.git'
            }
        }

        stage('Sonarqube Analysis') {
            steps {
                withSonarQubeEnv('sonar-server') {
                    sh """$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName='Dotnet-Webapp' \
                        -Dsonar.projectKey='Dotnet-Webapp'"""
                }
            }
        }

        stage('Quality Gate') {
            steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
                }
            }
        }

        stage('TRIVY File Scan') {
            steps {
                sh "trivy fs . > trivy-fs_report.txt"
            }
        }

        stage('OWASP Dependency Check') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --format XML', odcInstallation: 'DP-Check'
                archiveArtifacts artifacts: '**/dependency-check-report.xml', allowEmptyArchive: true
            }
        }

        stage('Docker Build & Tag') {
            steps {
                script {
                    withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                        sh "make image"
                    }
                }
            }
        }

        stage('TRIVY Image Scan') {
            steps {
                sh "trivy image nacromancer858/dotnet-monitoring:latest > trivy.txt"
            }
        }

        stage('Docker Push') {
            steps {
                script {
                    withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                        sh "make push"
                    }
                }
            }
        stage("Deploy to container"){
            steps{
                sh "docker run -d --name dotnet -p 5000:5000 nacromancer858/dotnet-monitoring:latest"
            } 
        }
       }
    }
}

Now to see the report, you can go to SonarQube Server and go to Projects.

Also, after running the above pipeline you can see your application on http:<PublicIP>:5000

Step 5: Setup Kubernetes (K8s)

• Go to cloned repo and then cd k8s_Instances/

• Run terraform init and then terraform apply --auto-approve

• It will create 2 instance One with the named Master and other with Worker.

• By default I have provided scripts into it. It will install all dependencies.

• LogIn into the Worker Node

• Run the following command:

sudo kubeadm join <master-node-ip>:<master-node-port> --token <token> --discovery-token-ca-cert-hash <hash>

• Now, LogIn into the Master Node

cd .kube/
cat config

• Copy the config file to Jenkins master or the local file manager and save it.

• Copy it and save it in documents or another folder save it as secret-file.txt

• Add K8s Credentials

• Go to manage Jenkins → manage credentials → Click on Jenkins global → add credentials

• The final step to deploy on the Kubernetes cluster, add this stage to the pipeline.

stage('Deploy to k8s'){
            steps{
                dir('K8S') {
                  withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
                    sh 'kubectl apply -f deployment.yaml'    
                   }
                }   
            }
        }

Before starting a new build remove old containers.

Output:

kubectl get svc
#copy service port 
<worker-ip:svc port>

You can see over there

Step 6: Destroy All the Infrastructure

• Go to your terminal, locate the directories and run:

    cd /k8s_Instances  
    terraform destroy --auto-approve
    cd /Instance_Terraform
    terraform destroy --auto-approve
  

Conclusion

By following the guide, organizations can streamline workflows using Docker containers and Kubernetes for scalability. Monitoring CPU performance offers insights into operational efficiency, enhancing deployment reliability and paving the way for resilient, scalable architectures in modern IT landscapes.