Application Deployment Project
Sushma EllaboinaThis blog is a step by step guide to show-case the project that was build to understand how application deployment works in real time situations.
Let’s understand it one by one.
Before deploying the application into Production, it has to undergo several Phases.
Developing Phase : This is when developers develop the code in their local, compile and run the code in their local, once everything works fine, push the committed code to GitHub. To develop the code in dev environment and verify if application is run in cloud same as in Local. They raise a Jira Ticket to Devops engineers to create server by providing specific CPU and Memory and run the application in cloud. Code is run multiple times by developer through a pipeline (through Webhooks/ Poll SCM) created by Devops Engineer until all bugs are fixed.
Testing/Building Phase : In this phase, a pipeline is created and provided to testing engineers to test all possible success/Failure scenarios of code. This is typically performed by Devops engineers. On success, this is signed off and moved to Release phase.
Release/Pre-Production/UAT Phase : Now that the application works as per the requirement, this is provided to Client to check if expectations are reached. On Success, they provide sign off on this and approve for release into Production.
Production/Deployment Phase : This is where the real application changes are done and Deployed. With this the changes are brought live and available to users.
Working of Pipelines :
We are going to deploy same application three times in different environments.
Develop : Develop the code —> Push the code to Git Hub —> Build the code —> Test —> Code Quality Analysis —>Deploy —> Post Build Actions.
Test : Copy the developed code —> Build —> Test—> Code Quality Analysis —> Artifact —> Deploy —> Post Build Actions.
Deploy ( Prod ) : Copy the Tested code —> Build —> Test —> Code Quality Analysis —> Artifact —>RM Approval —> Deploy —> Post Build Actions.
Tools Used :
Git : Code develop ( Source Code)
Maven : Build code + Unit Test
SonarQube (Port : 9000) : Code Quality Test
Jenkins ( Port : 8080) : Pipeline
Tomcat (Port : 8080): Application Deployment
Nexus (Port : 8081): Artifact
Email : Post Build Actions
Branches Used :
Develop Branch ( EC2 Instance : Dev - Env)
Feature Branch ( EC2 Instance : Test - Env) ( Push code to from Develop Branch to Feature Branch)
Release Branch ( EC2 Instance : UAT - Env)
Main Branch ( EC2 Instance : Prod - Env)
Develop —→ Feature —→ Release —→ Main
Servers to Create :
EC2 - Instance :: Tomcat(Dev - Env)
EC2 - Instance :: Jenkins
EC2 - Instance :: Nexus
EC2- Instance :: Sonar
EC2- Instance :: Tomcat (Test - Env)
EC2- Instance :: Tomcat (Prod - Env)
Procedure :
Create EC2 - Instance : Jenkins and EC2- Instance : Dev -Env and launch the Instance.
Create EC2 - Instance : SonarQube and Nexus and launch the instance ( Minimum 2 CPUs and 4 GB RAM and 25 GB of EBS Volume ; Instance type : C7iflex - large)
Install Jenkins in Jenkins Server.
Open Jenkins Server using public IPv4 and connect using ssh (From EC2 Instance).
Move to root user ( $sudo -i)
Change Hostname to Jenkins ($hostnamectl set-hostname “Jenkins”)
Run jenkins.sh script that will install Jenkins ( Take it from Git Hub → All Settings)
Install Tomcat in Dev-Env Server.
Open Dev-Env (Tomcat) Server using public IPv4 and connect using ssh (From EC2 Instance).
Move to root user ( $sudo -i)
Change Hostname to Dev-Env ($hostnamectl set-hostname “Dev-Env”)
Run tomcat.sh script that will install tomcat ( Take it from Git Hub → All Settings)
Install Nexus in Nexus Server.
Open Nexus Server using public IPv4 and connect using ssh (From EC2 Instance).
Move to root user ( $sudo -i)
Change Hostname to Nexus ($hostnamectl set-hostname “Nexus”)
Run nexus.sh script that will install nexus ( Take it from Git Hub → All Settings)
Install SonarQube in Sonar Server
Open Sonar Server using public IPv4 and connect using ssh (From EC2 Instance).
Move to root user ( $sudo -i)
Change Hostname to Sonar ($hostnamectl set-hostname “Sonar”)
Run first commands till line 13 directly in sonar.sh script that will install sonar ( Take it from Git Hub → All Settings)
Manually run last 2 commands ( If it doesn’t work, follow bellow commands)
$ cd /opt/sonarqube 8.9.10../bin/linux-x86-64
$ ll ( You will find file called : sonar.sh)
./sonar.sh start « To start sonar manually»
./sonar.sh status « To check status of sonar» ( It shows message as running)
Verify if all the required ports are present in each instance - configure port according to each server or select all traffic (Not recommended in real-time) by clicking edit inbound rules in instance. (add rule with custom port : 8080 -8081 and another rule with custom Port : 9000)
Set up Jenkins Dashboard
Copy and paste IP:8080 to browser.
Take the path displayed in first page and do ($cat <Path>) in Jenkins server.
Copy the token generated and paste in password section —> Click on Continue
Click on Install suggested plugins and fill in the details in the form —> Save and continue
Set up tomcat account.
Copy and paste IP:8080 to browser.
Click on Manager app —> Provide Username : tomcat, password : admin@123 in the pop up.
Set up Nexus account.
Copy and paste IP:8081 to browser.
Click on sign in over top-right corner —> username : admin, copy the path that is displayed and do ($cat <Path>) in Nexus server.
Copy the token generated and past in password section in nexus and click sign in.
Set a password for same —> click on next —> Click in disable anonymous access —> Next
Create a repository now —> Go to Settings —> Click on repositories —> create repository —> maven2(hosted) —> name : myrepo —> Deployment Policy : allow redeploy —> Click on create repository
Set up Sonarqube
Copy and paste IP:9000 to browser.
Default username and password of Sonarqube is : admin
Now set up new password —> Update
Note :: In real -time , If it is existing project, all this tools will be set up previously, but if it is new project, we need to set up all this tools.
Create a pipeline in Dev-env and provide this to Developers.
- Code Stage : New item → name: dev-pipeline → click on pipeline→ go to pipeline → click on try sample pipeline : Hello World → edit the pipeline → pipeline syntax → select git → Provide repository URL of Git Hub→ Branch ( dev-001 here) → Credentials ( In real- time we give private creds) ( here nothing)
pipeline {
agent any
stages {
stage('Code') {
steps{
« Paste script generated by pipeline syntax»
} } } }
Create a branch ( called : dev-001) in Git hub and give that to developers - they will push the code to this branch and perform commits.
Build Stage : Configure Mavin :: go to Manage Jenkins → click on tools → click on add maven → give name : mymaven → Save
Use this maven in pipeline —> go to pipeline → Configure → give tools section after agent any and also add Build stage and add the command clean package.
pipeline {
agent any
tools {
maven “mymaven”
}
stages {
stage('Code') {
steps {
«Paste Pipeline syntax»
}
}
stage (‘Build‘) {
steps {sh ‘mvn clean package’
}
}
} }
go to tomcat ( dev-env ) server and add java - this is considered as slave ( Java already installed when tomcat is installed) [ Adding Server ]
Install git in dev-env server ( $ yum install git -y)
Create a node : Manage Jenkins → Node → Create Node → Node name : dev-slave → Permanent → Create → description → No-of executors :2 → remote root directory : /root/ec2-user/jenkins/ → Usage : only build jboss….. → Launch Method : Launch via SSH → Host : « Private IP of dev-env server» → add credentials → kind : SSH username with private key → Id , Description, username : ec2-user —> Private Key → Enter directly→ ( Copy the private key from .pem file that was downloaded) → click on add and select the credentials. → Host key : Non verifying verification strategy → Save
You will get the node created (dev-node ) into running, if not, run disk space scripts to clear space. - Slave is now ready.
Add node after agent in pipeline as well.
pipeline {
agent {
node {
label ‘dev’
}
}
tools {
maven “mymaven”
}
stages {
stage('Code') {
steps {
«Paste Pipeline syntax»
}
}
stage (‘Build‘) {
steps {sh ‘mvn clean package’
}
}
} }
Since this needs to be run by Developer, provide access to developer ( User Access Management )
Add User : Go to manage jenkins→ User → Create user→ username : avinash , password, full name, email → Create
User Permissions : Go to manage jenkins again → Security →Under authorization, select Project-based matrix Authorization strategy → add user → By default admin should have full permissions (i.e. I should have check box ticked for Overall) and developer : avinash should be checked with : read under overall section —> Save
Job Permissions : Go to pipleline (dev -pipeline)→ check Enable Project based security → add user : avinash , give job read , job build , job cancel permissions.
Provide the URL to Developer avinash now and he can access according to permissions given by us.
Before building, for better pipeline view, install a plugin → Manage jenkins → Plugin → Pipeline : Stage view → Install
Developer clicks on Build now and can see successful builds in Pipeline.
Till now we did only 2 stages Code and Build. Now, lets do Code Quality Analysis.
CQA Stage : Go to Sonarqube → Click on add Project →Create Project (give name: eg : Java-Project) → set up→ Generate token give any token name→ You get token displayed → copy that to notepad and continue
In next step, click on maven displayed → copy that command generated and past in pipeline, but before that create sonarqube plugin.
Go to Jenkins → manage jenkins → Plugin → available plugin → select Sonarqube Scanner plugin → Install
Integrate Sonar with Jenkins : Visit official documentation if not aware and select sonar scanner for Jenkins → Read steps of Installation i.e. Go to Manage Jenkins → system → scroll down to get sonarqube servers → add sonarqube→ give any name (eg: mysonar)→ Give Sonar URL → Under sonar authentication token → add ( We are integrating sonar with token and not with username , password) → so select secret text under kind → Paste token copied to notepad in Secret → id,description→ add
Go to Manage Jenkins → tools → click on add sonarqube scanner → give name : mysonar → save.
Now go to our pipeline →Configure → add the sonar name in the pipeline .
Also, add CQA stage and provide the maven commands that were generated in sonarqube tool.
pipeline {
agent {
node {
label ‘dev’
}
}
environment {
SCANNER_HOME = “mysonar”
}
tools {
maven “mymaven”
}
stages {
stage('Code') {
steps {
«Paste Pipeline syntax»
}
}
stage (‘Build‘) {
steps {sh ‘mvn clean package’
}
}
stage (‘CQA‘){
steps {
sh ‘‘‘
< Paste copied commands from sonar>
‘‘‘
}
}
} }
I will give this pipeline to Developer now and they will run the pipeline.
Developers can : Go to Sonarqube dashboard and select the project that was created ( eg :Java-project ), you will find all the bugs , its count, and also the recommendations on what the issue is and where the issue is observed - file/path and line number ( eg : Use <em> instead of <i>
After correcting the bugs, developer will commit the code and run the pipeline again.
Since we cannot keep building this code again and again manually, once any changes are performed and committed by developer, pipeline will run automatically ( with the help of Webhooks or Poll SCM)
Go to Github → repository settings → Webhooks → add Webhooks → Pay load URL : «Jenkins URL»/github-webhooks → content-type : application/json → add webhook
Go to Jenkins → select pipeline→ Configuration→ triggers → check Github hook trigger → save ( Now web hooks will be activated in Git hub : through message successful)
Now pipeline will be triggered automatically everytime developer commits the code.
Deploy Stage : Deployment to be done in tomcat server.
Go to Manage jenkins → Plugin→ available plugin → Deploy to Container → Install → Check Restart jenkins box instead of going to server and restart.
Now go to pipeline → configure → Pipeline syntax → Select Deploy war/ear to Container → War/Ear files : target/*.war → Context path : e-commerce → add container : tomcat 9.x Remote → add credentials → Kind : username with password → Give tomcat username and password, id, description→ add → select credentials -. provide tomcat URL → click on generate pipeline script and copy the script.
Create new stage : deploy and paste the script in it.
pipeline {
agent {
node {
label ‘dev’
}
}
environment {
SCANNER_HOME = “mysonar”
}
tools {
maven “mymaven”
}
stages {
stage('Code') {
steps {
«Paste Pipeline syntax»
}
}
stage (‘Build‘) {
steps {sh ‘mvn clean package’
}
}
stage (‘CQA‘){
steps {
sh ‘‘‘
< Paste copied commands from sonar>
‘‘‘
}
}
stage (‘deploy’){
steps {
< Paste copied pipeline script >
}
}
} }
Save and run the build → now when i refresh the tomcat, e-commerce application is deployed.
*** Now Merge develop branch into feature branch in Github for testing ***
Click on Compare and pull request → Create pull request → merge ( feature ← develop ) → confirm merge
- Create a pipeline in test-env and provide this to test engineers/devops engineers.
For that we are taking feature branch and build the run in test environment.
Create another server ( EC2- Instance : Test -env) and connect it.
Install tomcat in Test-env server [ Note : whichever server or environment(dev/test/prod) you want to run the application, install tomcat always]
Build slave for test-server i.e. create node with test -slave.
To create new pipeline for test, we don’t need to create again from scratch, we can copy from dev pipeline. New item → test-pipeline→ copy from : <dev-pipeline>
We get dev pipeline code now, everything remains same, except label name: «test», branch name : «Feature» , Tomcat URL.
Now create a new user (tester )and provide access and permissions.
By above pipeline, all develop,build,test,deploy stages are completed and we are to add artifact.
Artifact Stage :
Go to manage Jenkins → plugin → Nexus Artifact uploader → Install ( No need to restart for nexus)
Go to the pipeline ( test -pipeline ) → pipeline syntax → select Nexus Artifact Uploader → version : Nexus 3 → Protocol : HTTP → Nexus URL → add credentials → username : admin , Password : Nexus Password → id, description →add → select credentials → group id : «Take it from pom.xml» → version → repository : «Repository created in sonar i.e. myrepo» → add artifact → artifact id : «from pom.xml» → type : war → File : target/myweb-8.6.9.war → click on generate pipeline syntax and copy the script.
Now add another stage : artifact in pipeline.
pipeline {
agent {
node {
label ‘dev’
}
}
environment {
SCANNER_HOME = “mysonar”
}
tools {
maven “mymaven”
}
stages {
stage('Code') {
steps {
«Paste Pipeline syntax»
}
}
stage (‘Build‘) {
steps {sh ‘mvn clean package’
}
}
stage (‘CQA‘){
steps {
sh ‘‘‘
< Paste copied commands from sonar>
‘‘‘
}
}
stage (‘deploy’){
steps {
< Paste copied pipeline script >
}
}
stage (‘artifact’){
steps {
< Paste copied pipeline script >
}
}
} }
Now save it and build. This is run by devops engineer. we can verify by going to sonar and you can find the war file there and application is deployed in tomcat for test.
Create a pipeline for Production.
Create another server ( EC2- Instance : Prod -env)
Repeat Same steps as test -env by copying test-pipeline under main branch and add extra RM approval in pipeline.
Subscribe to my newsletter
Read articles from Sushma Ellaboina directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by