Automated CI/CD Pipeline for Django Web Application using AWS, Docker, Jenkins and Github.

Project Description:

This project involves automating the deployment process of a Django Notes application using Jenkins and Docker. It includes setting up Jenkins and Docker, creating a Jenkins declarative pipeline, integrating with GitHub, and implementing stages for code checkout, building, testing, and deployment. The pipeline ensures efficient and reproducible software delivery, allowing for continuous integration and deployment of the application.

Step 1:Launch an EC2 instance with the below Jenkins or Docker installation script.

Script.sh

#!/bin/bash
sudo yum update -y
sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
sudo yum upgrade -y
sudo amazon-linux-extras install java-openjdk11 -y
sudo yum install jenkins -y
sudo systemctl enable jenkins
sudo systemctl start jenkins

# Install Docker
sudo amazon-linux-extras install docker -y
sudo service docker start
sudo usermod -aG docker ec2-user
sudo usermod -aG docker jenkins


# Enable Docker on system boot
sudo systemctl enable docker

Check status of jenkins/docker if running or not

Once done with installations, open port 8080 in the security group

Step 2: Try to access public IP 44.201.141.94:8080

Unlock Jenkins with the below key:

Create credentials for a first-time user

Step 3:generate keys by ssh-keygen

This will generate public and private keys on the machine.

Id_rsa: Private Key

Id_rsa.pub: Public Key

Step 4:Navigate to manage jenkins and under available plugins, search for github Integration, as we need this plugin for the integration with Github (our code is on github)

Step 5:Now move back to our problem statement for project

Select new item like the one below

Declarative Pipeline is a more recent feature of Jenkins Pipeline, which:

• provides richer syntactical features over Scripted Pipeline syntax, and

• is designed to make writing and reading Pipeline code easier.

Why Pipeline?

Jenkins Pipeline is a powerful automation tool that allows you to define your software delivery pipeline as code. It provides a way to express and visualize your entire software delivery process, from building and testing to deploying and releasing.

Using Jenkins Pipeline, you can define your pipeline as a series of stages and steps that are written in code and stored in a version control system. This brings several benefits:

1. Reproducibility and Maintainability: By defining your pipeline as code, you have a reproducible and version-controlled representation of your entire delivery process. This makes it easier to manage and maintain changes to your pipeline over time.

2. Flexibility and Customization: Jenkins Pipeline provides a wide range of features and constructs, allowing you to customize and adapt your pipeline to meet the specific needs of your project. You can define parallel execution, conditional steps, error handling, and more.

3. Visibility and Collaboration: With Jenkins Pipeline, your pipeline definition is visible to the entire team. This enables better collaboration, as everyone can understand and contribute to the pipeline. It also facilitates code reviews and ensures that the pipeline is a shared and well-documented artifact.

4. Portability: Jenkins Pipeline is not tied to any specific technology stack or platform. It can be used for building, testing, and deploying applications in any language or on any platform. This makes it a versatile tool that can be adapted to different projects and environments.

Jenkins Pipeline provides a structured and flexible way to define, visualize, and automate your software delivery process. It brings your pipeline configuration into code, making it easier to manage, maintain, and collaborate on your delivery process.

Step6:Declarative pipeline:

Step 7:Using credentials: As we used Docker Hub to push the image,

There are numerous third-party sites and applications that can interact with Jenkins, for example, artifact repositories, cloud-based storage systems and services, and so on.

Jenkins can store the following types of credentials:

• Secret text: a token such as an API token (e.g. a GitHub personal access token),

• Username and password, which could be handled as separate components or as a colon-separated string in the format username:password (read more about this in Handling credentials),

• A secret file, which is essentially secret content in a file,

• SSH Username with Private Key: An SSH public/private key pair

• Certificate: a PKCS#12 certificate file and optional password, or

• Docker Host Certificate Authentication credentials

So here we will use Docker Hub to login and push the images.

Manage jenkins-->>credentials-->>put dockerhubcredentials(username/password)

Step 8: The code is available on GitHub. copy code url and put under project url

Step 9:How will the stages look?

Four stages:

Stage 1: Code (provided giturl as code is present on github),

Stage 2: Build and Test Build an image from dockerfile

Python 3.9 is the version—Docker images can inherit from other images. Therefore, instead of creating your own base image, you can use the official Python image, which has all the tools and packages needed to run a Python application.

Workdir /app/backend: To make things easier when running the remaining commands, create a working directory. This instructs Docker to use this path as the default location for all subsequent commands. This means you can use relative file paths based on the working directory instead of full file paths.

TheCOPY command takes two parameters. The first parameter tells Docker what file(s) you would like to copy into the image. The second parameter tells Docker where to copy those files. For this example, copy the requirements.txt file into the working directory /app.

COPY requirements.txt /app/backend

With therequirements.txt file inside the image, you can use the RUN command to run pip3 install. This works exactly the same as runningpip3 install locally on a machine, but this time Pip installs the modules into the image.

RUN pip3 install -r requirements.txt

At this point, you have an image based on Python version 3.9 and have installed the dependencies. The next step is to add the source code to the image. Use theCOPY command as with the requirements.txt file. ThisCOPY command takes all the files located in the current directory and copies them into the image.

COPY /app/backend

Now, tell Docker what command to run when the image is executed inside a container using the CMD command. Note that you need to make the application externally visible (i.e. from outside the container) by specifying --runserver

Stage 3: Login and Push Image: As credentials can be misused or there are policy violations to put credentials under code, Jenkins provides to put credentials under global credential and use them in code with this syntax so that misuse of credentials can be overcome.

Syntax:withCredentials([usernamePassword(credentialsId:'dockerhub', passwordVariable:'dockerhubpassword', usernameVariable:'dockerhubuser')]) { sh "docker login -u ${env.dockerhubuser} -p ${env.dockerhubpassword}" sh 'docker push gajananbarure/django-notes-app:latest'

Stage 4: Deploy: The final step in the deployment of our Django notes app, which we can use to write-out the notes,

Pipeline:

pipeline {
    agent any

    stages{
        stage('Code'){
            steps{
                git url: 'https://github.com/Omkar0070/django-notes-app.git', branch: 'main' 
            }
        }
        stage('Build and Test'){
            steps{
                sh 'docker build . -t omkarkadam0070/django-notes-app:latest'
            }
        }
        stage('Login and Push Image'){
            steps {
                echo 'login in docker hub and pushing'
                withCredentials([usernamePassword(credentialsId:'dockerhub', passwordVariable:'dockerhubpassword', usernameVariable:'dockerhubuser')]) {
                                  sh "docker login -u ${env.dockerhubuser} -p ${env.dockerhubpassword}"
                                  sh 'docker push omkarkadam0070/django-notes-app:latest'
                }
            }
        }
        stage('Deploy'){
            steps{
                sh "docker run -d -p 8000:8000 django-notes-app"
            }
        }
    }
 }

Step10:Pipeline successful..!!

Step 11:Image pushed to Docker Hub

Access Public IP with port 8000

Django:notes app deployed successfully by Declarative Pipeline

Step 12:Now lets move to the CICD part

This is to autotrigger the builds if any changes in code we do not have to trigger manually.

cd.ssh -->>ssh-keygen-->>Put this key in: root@ip-172-31-92-121:/.ssh$ cat id_rsa.pub

Setting-->> and generating a personal access token, which we can provide in Jenkins under secret text.

Step 13:Add this secret key to Jenkins undersecret.

Select this option to trigger the build after code change in git. It will reflect

Step 14:Navigate to setting-->>webhook-->>3.83.136.98:8080/github-webhook

Webhooks can be triggered whenever specific events occur on GitHub. For example, you can configure a webhook to trigger whenever Code is pushed to a repository.

Make sure it is successful, as it will show a green arrow.

Step 15:Make some changes in the code and see if the build gets triggered or not.

Code availability:https://github.com/Omkar0070/django-notes-app