→ Jenkins is an open-source automation server that accelerates software development. It orchestrates the entire software delivery pipeline from building and testing to deployment.

Jobs: Jobs are building blocks of the pipeline. Each jo defines a specific task like compiling code, running tests, or deploying to a specific environment. Each execution of a job is considered a build.

Builds: Jenkins maintain a build history, allowing us to track past runs which could be either successful or failed for troubleshooting purposes.

Freestyle project: The default project type, offering flexible job definations.

Pipelines: Automate workflows that orchestrate jobs from code build to deployment. These are define as code in a Jenkinsfile, enabling version control and collaboration.

Stages: Within a pipeline, we can group related jobs into stages. This provides a clear structure for the workflow like build, test, deploy, making it easier to visualize different phases of a pipeline.

Nodes: Machines where Jenkins executes jobs, with a single controller (Jenkins controller) or multiple agents.

Plugins: Extend Jenkins’ functionality, integrating with various tools and technologies.

What is CI/CD ?

Continuous Integration (CI) and Continuous Delivery (CD) are practices that enable software development teams to deliver code changes more frequently and reliably. Let's consider a scenario to illustrate how CI/CD works:

Scenario: Developing a Web Application

1. Continuous Integration (CI):

   • Code Integration: Developers work on different features or bug fixes in separate branches of a version control system (e.g., Git). When a developer completes a feature, they merge their code into the main branch.

   • Automated Testing: Once the code is merged, an automated build process is triggered. Jenkins, for example, can be used to automatically compile the code and run a suite of automated tests (unit tests, integration tests) to ensure that the new code does not break existing functionality.

   • Feedback Loop: If any tests fail, the CI system immediately notifies the developer, allowing them to address issues quickly. This rapid feedback loop helps maintain code quality and reduces integration problems.

2. Continuous Delivery (CD):

   • Staging Environment: After successful integration and testing, the application is automatically deployed to a staging environment. This environment closely mirrors the production environment and allows for further testing, such as user acceptance testing (UAT).

   • Automated Deployment: The deployment process is automated, ensuring that the application can be deployed consistently and reliably. Scripts or tools manage the deployment, reducing the risk of human error.

   • Approval and Release: Once the application passes all tests in the staging environment, it is ready for release. In a Continuous Delivery setup, the deployment to production is a manual step that requires approval. However, the deployment process itself is automated.

3. Continuous Deployment (optional):

   • In some cases, teams adopt Continuous Deployment, where every change that passes automated tests is automatically deployed to production without manual approval. This approach requires a high level of confidence in the automated testing and deployment processes.

By implementing CI/CD, the development team can ensure that code changes are integrated and delivered to users quickly and with high quality, reducing the time to market and improving the overall development workflow.

Jenkins architechture

Jenkins architecture supports distributed builds and efficient workflow management. Key components include:

1. Jenkins Controller (Master): Manages the build environment, schedules jobs, and monitors execution on agents. It provides the web interface for configuration and results.

2. Jenkins Agents (Slaves): Machines that perform the actual build work, managed by the controller, enabling distributed builds and load balancing.

3. Jobs: Fundamental units of work, such as building code or running tests, configured to run on specific or any available agents.

4. Builds: Each job execution is a build, with Jenkins maintaining a history for tracking and troubleshooting.

5. Pipelines: Define complex workflows as code in a Jenkinsfile, automating the software delivery process with stages and parallel execution.

6. Plugins: Extend Jenkins' functionality, integrating with various tools and technologies for diverse use cases.

7. User Interface: Web-based interface for configuring jobs, monitoring builds, and managing the environment, providing access to results and logs.

This architecture ensures Jenkins is scalable and flexible, supporting various development and deployment scenarios.

What is Nodes in jenkins ?

In Jenkins, nodes refer to the machines where Jenkins executes jobs. There are two main types of nodes:

1. Jenkins Controller (Master): This is the central node that manages the build environment, schedules jobs, and distributes tasks to agents. It also provides the web interface for users to interact with Jenkins.

2. Jenkins Agents (Slaves): These are additional nodes that perform the actual work of executing jobs. Agents can be set up on different machines and platforms, allowing Jenkins to distribute the workload and perform builds in parallel. This setup enables scalability and efficient resource utilization.

Nodes are essential for Jenkins' distributed architecture, allowing it to handle multiple jobs simultaneously and manage large-scale build environments.

What is agent in jenkins ?

In Jenkins, an agent (also known as a slave) is a machine that performs the actual work of executing jobs. Agents are part of Jenkins' distributed architecture, allowing the workload to be spread across multiple machines. This setup enables Jenkins to handle multiple jobs simultaneously and efficiently utilize resources. Agents can be configured on different machines and platforms, providing flexibility and scalability in managing build environments. They are managed by the Jenkins controller (master), which schedules jobs and assigns them to available agents for execution.

Explain the wrokflow between controller node and worker nodes.

In Jenkins, the workflow between the controller node (master) and worker nodes (agents) is essential for executing jobs efficiently. Here's how it typically works:

1. Job Configuration: Users configure jobs on the Jenkins controller through the web interface. These jobs define tasks such as building code, running tests, or deploying applications.

2. Job Scheduling: When a job is triggered (either manually or automatically), the Jenkins controller schedules it for execution. The controller determines which agent is suitable for running the job based on factors like labels, availability, and resource requirements.

3. Task Assignment: The controller assigns the job to an available agent. This assignment is based on the configuration and the current load on the agents.

4. Job Execution: The assigned agent executes the job. It performs the tasks defined in the job configuration, such as compiling code, running tests, or deploying artifacts.

5. Monitoring and Logging: During execution, the agent sends logs and status updates back to the controller. Users can monitor the progress and view logs in real-time through the Jenkins web interface.

6. Completion and Reporting: Once the job is completed, the agent reports the results back to the controller. The controller updates the job status (success or failure) and stores the build history for future reference.

7. Resource Management: The controller manages resources by distributing jobs across multiple agents, ensuring efficient utilization and load balancing.

This workflow allows Jenkins to handle multiple jobs simultaneously, leveraging distributed resources to optimise performance and scalability.

Installation & Configure

→ If we install Jenkins using system packages on Linux, the jenkins home is usually set to /var/lib/jenkins directory. The Jenkins home directory is a crucial part of a Jenkins setup, which is where Jenkins stores all its data and configurations. This includes jobs, plugins, configurations, build logs, artifacts archives, and other metadata related to the Jenkins server.

We can install Jenkins by visiting https://www.jenkins.io/doc/book/installing/linux/#debianubuntu

To check the status of the Jenkins.

systemctl status jenkins

To check the status of jenkins in details when there are some error.

journalctl -u jenkins

Start Jenkins

You can enable the Jenkins service to start at boot with the command:

sudo systemctl enable jenkins

You can start the Jenkins service with the command:

sudo systemctl start jenkins

You can check the status of the Jenkins service using the command:

sudo systemctl status jenkins

Password

We can get Jenkins password by going to the end of the log using journalctl -u jenkins or by visiting /var/lib/jenkins/secrets/initialAdminPassword

Login

By default jenkins use localhost:8080

We can change the port by visiting sudo vi /lib/systemd/system/jenkins.service

Configure Sudo to Allow Jenkins User to Run Commands Without a Password

sudo visudo #Open the sudoers file for editing using visudo (to ensure you don't make any syntax mistakes)
jenkins ALL=(ALL) NOPASSWD: ALL #Add the following line to grant the Jenkins user (assuming the Jenkins user is jenkins) permission to run commands without entering a password

Types of Jenkins projects

In Jenkins, there are several types of projects (also known as job types) that you can create depending on your needs. Here's a breakdown of the most common Jenkins project types:

---

1. Freestyle Project

• Description: The most basic and flexible project type in Jenkins.

• Use cases: Simple build/test/deploy jobs.

• Key features:

  • Configure build steps and post-build actions using a GUI.

  • Supports SCM integrations (e.g., Git), build triggers (e.g., cron, Git hook), and multiple build steps.

---

2. Pipeline Project

• Description: Uses a Jenkinsfile written in Groovy to define the build pipeline as code.

• Use cases: Complex workflows with multiple stages, parallelism, and better control over execution.

• Key features:

  • Supports Declarative and Scripted pipelines.

  • Stored as code (in SCM).

  • Supports stages, steps, environment variables, etc.

---

3. Multibranch Pipeline Project

• Description: Automatically creates a pipeline for each branch in your source control repository.

• Use cases: CI/CD for multiple Git branches (e.g., feature branches).

• Key features:

  • Jenkins scans the repo and creates jobs for each branch.

  • Automatically discovers and uses Jenkinsfile from each branch.

---

4. Folder

• Description: Used to group multiple jobs or pipelines.

• Use cases: Organizing jobs by project, team, or environment.

• Key features:

  • Access control at the folder level.

  • Easier navigation in large Jenkins instances.

---

5. GitHub Organization Project

• Description: Automatically scans a GitHub organization for repositories and creates jobs.

• Use cases: Managing CI/CD for all repositories in a GitHub org.

• Key features:

  • Uses GitHub API to discover repositories.

  • Automatically configures multibranch pipeline jobs per repo.

---

6. Maven Project

• Description: Specialized support for building Maven-based Java projects.

• Use cases: Java projects using Apache Maven.

• Key features:

  • Automatic detection of build steps based on pom.xml.

  • Built-in support for reporting test results and code coverage.

---

7. External Job

• Description: Tracks execution of jobs run outside Jenkins (manually or from another system).

• Use cases: Legacy systems or external tools not directly run by Jenkins.

• Key features:

  • Minimal Jenkins involvement.

  • Used more for job result tracking.

---

8. Matrix Project (Deprecated in some setups)

• Description: Allows running the same job across multiple environments (e.g., OS, JDK version).

• Use cases: Testing software on multiple platforms or configurations.

• Key features:

  • Define axes (variables) and run builds for all combinations.

  • Powerful but complex; often replaced by pipeline jobs with parallel stages.

---

Summary Table:

---

We can add the PATH variable, both ways; such as hardcoded and by setting global variables in Jenkins (as required for the job)

What is Plugins in Jenkins?

Plugins in Jenkins are extensions that enhance its functionality by integrating with various tools and technologies. They allow Jenkins to support a wide range of features and capabilities, such as source control management, build tools, user interface improvements, and more. The extensive plugin ecosystem enables Jenkins to be highly customizable and adaptable to different development and deployment needs. Plugins can be installed and managed through the Jenkins web interface, making it easy to extend Jenkins' capabilities without modifying its core code.

Example

yet another build visualizer A plugin to visualise the flow of jobs

What is controller failure?

In Jenkins, a controller failure refers to a situation where the Jenkins controller (also known as the master) experiences an issue that prevents it from functioning properly. The controller is responsible for managing the build environment, scheduling jobs, and distributing tasks to agents. A failure can occur due to various reasons such as hardware malfunctions, software bugs, resource exhaustion, or network issues. When a controller failure happens, it can disrupt the entire Jenkins environment, affecting job scheduling and execution. To mitigate such failures, it's important to implement backup and recovery strategies, monitor system health, and ensure high availability configurations.

Freestyle and pipeline projects in Jenkins are both affected by the overall performance and stability of the Jenkins environment, including any issues such as controller failures.

• Freestyle Projects: These are simpler and more flexible, allowing users to configure build steps and post-build actions through a graphical user interface. They can be impacted by system performance issues, as they rely on the Jenkins controller to manage job execution and resource allocation.

• Pipeline Projects: These are defined as code in a Jenkinsfile, allowing for more complex workflows with multiple stages and parallel execution. Pipeline projects are more resilient to certain types of failures due to their ability to define complex error handling and recovery steps within the pipeline script itself. However, they still depend on the Jenkins controller for scheduling and managing the execution of jobs across agents.

Both project types require a stable Jenkins environment to function optimally, and any disruptions can affect their execution and the overall CI/CD process.

What is fingerprints in Jenkins?

In Jenkins, fingerprints uniquely identify files to track their usage and movement across jobs and builds, aiding in artifact tracing, consistency, and dependency management.

Jenkins Chained Project

Question

Create two new freestyle projects

1. ascii-build-job

2. ascii-test-job (this step should get triggered when ascii-build-job is successful)

Below is a sample shell script to use, although you are free to use any script of your choice.

ascii-build-job:

#!/bin/bash

# Fetch advice from the API
curl -s https://api.adviceslip.com/advice > advice.json

# Extract advice text
cat advice.json

ascii-test-job :

#!/bin/bash
ls advice.json
cat advice.json | jq -r .slip.advice > advice.message
# Test to make sure the advice message has more than 5 words
word_count=$(wc -w < advice.message)

if [ $word_count -gt 5 ]; then
  echo "Advice has more than 5 words"
else
  echo "Advice $(cat advice.message) has 5 words or less"
fi

Note: The ascii-test-job pipeline will show error msg in logs as it doesn't have the advice.json artifact, we will solve it using copy artifact.

Answer

Steps to Create and Configure Jenkins Jobs

1. Create ascii-build-job

1. Open Jenkins and click on "New Item".

2. Enter "ascii-build-job" as the name and select Freestyle project.

3. Click OK.

2. Configure ascii-build-job

• In the Build Steps, click Add build step → Execute shell.

• Copy and paste the following script:

#!/bin/bash

# Fetch advice from the API
curl -s https://api.adviceslip.com/advice > advice.json

# Extract advice text
cat advice.json

• Click Save.

3. Create ascii-test-job

1. Go back to Jenkins Dashboard and click New Item.

2. Enter "ascii-test-job" as the name and select Freestyle project.

3. Click OK.

4. Configure ascii-test-job

• In the General section, check "This project is parameterized".

• Add a File Parameter:

  • Name: advice.json

  • Description: "Advice file from ascii-build-job"

• In the Build Steps, click Add build step → Execute shell.

• Copy and paste the following script:

#!/bin/bash

ls advice.json
cat advice.json | jq -r .slip.advice > advice.message

# Test to make sure the advice message has more than 5 words
word_count=$(wc -w < advice.message)

if [ $word_count -gt 5 ]; then
  echo "Advice has more than 5 words"
else
  echo "Advice $(cat advice.message) has 5 words or less"
fi

• Click Save.

5. Set Up Post-Build Trigger in ascii-build-job

• Open ascii-build-job.

• Click Configure.

• Scroll to Post-build Actions and click "Add post-build action" → Build other projects.

• Enter ascii-test-job in the Projects to build field.

• Choose "Trigger only if build is stable".

• Click Save.

What is Pipeline project and Jenkins file?

A Pipeline project in Jenkins is a way to define and automate complex workflows and processes as code. It allows you to create a series of steps, known as a pipeline, that can include building, testing, and deploying applications. This approach provides better control over the execution process, supports parallel execution, and allows for more complex error handling and recovery steps.

A Jenkinsfile is a text file that contains the definition of a Jenkins Pipeline. It is written in Groovy and can be stored in the source control repository alongside the code it builds. This enables version control and collaboration on the pipeline itself. The Jenkinsfile defines the stages and steps of the pipeline, allowing for a clear and structured workflow that can be easily maintained and updated.

Pipeline vs Freestyle

Jenkins Pipeline with ENV using SCM

Q. Task: Create a Jenkins Pipeline for a Python Project use env variables

Scenario:
Please set up a Jenkins pipeline named python-pipeline to integrate with a Git repository that contains a sample project, such as a Python application. Ensure that the Jenkins SCM plugin is installed and configure the necessary parameters, including BRANCH_NAME and RUN_TESTS. Additionally, enable a webhook in the Git repository to trigger the build and test pipeline from SCM. The pipeline should be configured to perform the following steps:

1. Checkout the specified branch.

2. Install dependencies.

3. Run tests based on the RUN_TESTS parameter.

4. Build or package the project.

Hint:

Please create a Jenkinsfile at the location /root/python-app and push it to the repository (Code-Repo) http://git-server:3000/max/node-app.git (Not workable for you)

Furthermore, in the Jenkins UI, create a pipeline named python-pipeline and input the necessary details.

Answer:

Add the following Jenkinsfile at the path /root/python-app:

pipeline {
    agent any

    environment {
        REPO_URL = "http://git-server:3000/max/python-app.git"
        REPO_DIR = "${WORKSPACE}/python-app"  // Use Jenkins workspace directory
        BRANCH_NAME = "main"
        RUN_TESTS = "true"
        PYTHON_VERSION = "python3"
    }

    stages {
        stage('Checkout Code') {
            steps {
                script {
                    echo "Cloning branch: ${BRANCH_NAME} from ${REPO_URL}"
                    sh """
                        if [ -d "${REPO_DIR}" ]; then
                            cd ${REPO_DIR}
                            git reset --hard
                            git checkout ${BRANCH_NAME}
                            git pull origin ${BRANCH_NAME}
                        else
                            git clone --branch ${BRANCH_NAME} ${REPO_URL} ${REPO_DIR}
                        fi
                    """
                }
            }
        }

        stage('Install Dependencies') {
            steps {
                script {
                    echo "Installing dependencies..."
                    dir("${REPO_DIR}") {
                        sh """
                            set -e
                            ${PYTHON_VERSION} -m venv venv
                            . venv/bin/activate
                            pip install --upgrade pip
                            pip install -r requirements.txt
                        """
                    }
                }
            }
        }

        stage('Run Tests') {
            when {
                expression { return env.RUN_TESTS == "true" }
            }
            steps {
                script {
                    echo "Running tests..."
                    dir("${REPO_DIR}") {
                        sh """
                            set -e
                            . venv/bin/activate
                            pytest --junitxml=reports/test-results.xml
                        """
                    }
                }
            }
        }

        stage('Run Application') {
            steps {
                script {
                    echo "Starting Flask app for 60 seconds..."
                    dir("${REPO_DIR}") {
                        sh '''
                            set -e
                            . venv/bin/activate
                            echo "Flask app will run for 60 seconds..."
                            timeout 60s python app.py || echo "App terminated after timeout, ignoring error"
                        '''
                    }
                }
            }
        }
    }

    post {
        success {
            echo "Pipeline executed successfully!"
        }
        failure {
            echo "Pipeline failed!"
        }
    }
}

Once added, use the following commands to push to the Git repository:

git add .
git commit -m "Create and Push Jenkinsfile"
git push --set-upstream origin main

Next, navigate to the Jenkins UI to create a new Pipeline named python-pipeline. Proceed with the following steps:

1. Click on the Pipeline section.

2. Under Definition, select Pipeline Script from SCM.

3. Set SCM to Git.

4. Input the Repository URL as http://git-server:3000/max/python-app.git (this is the Code-Repo which we have already configured and can be accessed at top right of page).

5. Scroll down to the Branches to build section and specify the branch as */main.

6. Add Script Path as Jenkinsfile.

7. Click the SAVE button.

Finally, initiate the pipeline by selecting the Build Now option on the left side of the Jenkins UI.

Jenkins Declarative and Scripted Pipeline

Task: Create a Declarative Jenkins Pipeline for a Node.js Project

Scenario:
You are tasked with automating the deployment of a Node.js application stored in a Git repository (which can be found at top right Code-Repo) using a Declarative Jenkins Pipeline named declarative-pipeline. The following steps must be implemented:

1. Install the necessary dependencies with npm install.

2. Launch the application using node app.js.

Hint:

Create a Jenkinsfile named Jenkinsfile-declarative at location /root/node-app and push it to the repository (Code-Repo) http://git-server:3000/max/node-app.git

Furthermore, in the Jenkins UI, create a pipeline named declarative-pipeline and input the necessary details.

Answer:

Create a Jenkinsfile named Jenkinsfile-declarative at location /root/node-app include the following :

pipeline {
    agent any

    stages {

        stage('Checkout Code') {
            steps {
                script {
                    checkout scm
                }
            }
        }

        stage('Install Dependencies') {
            steps {
                script {
                    sh 'npm install'
                }
            }
        }

        stage('Run App') {
            steps {
                script {
                    // Stop any previous instance of the app
                    sh 'pkill -f "node app.js" || true'

                    // Start the app in the background and redirect logs
                    sh 'nohup node app.js > app.log 2>&1 &'

                    // Wait a few seconds to ensure the app starts
                    sleep 60

                    // Check if the app is running
                    sh 'curl http://localhost:3001 || echo "App is not responding!"'
                }
            }
        }
    }

    post {
        always {
            echo 'Cleaning up...'
        }
    }
}

Once added, use the following commands to push to the Git repository:

git add .
git commit -m "Create and Push Jenkinsfile-declarative"
git push --set-upstream origin main

Next, navigate to the Jenkins UI to create a new Pipeline named declarative-pipeline. Proceed with the following steps:

1. Click on the Pipeline section.

2. Under Definition, select Pipeline Script from SCM.

3. Set SCM to Git.

4. Input the Repository URL as http://git-server:3000/max/node-app.git (this is the Code-Repo which we have already configured and can be accessed at top right of page).

5. Scroll down to the Branches to build section and specify the branch as */main.

6. Add Script Path as Jenkinsfile-declarative.

7. Click the SAVE button.

Finally, initiate the pipeline by selecting the Build Now option on the left side of the Jenkins UI.

Autometion and security

Jenkins offers two powerfull options for oautometing the tasks, Jenkins CLI, and Jenkins Rest API.

Jenkins CLI

Jenkins provides a built-in CLI that enables users and administrators to interact with Jenkins from a script or shell environment.

• We can access the Jenkins CLI over SSH or by using the Jenkins CLI client - JAR.

• The CLI client is distributed as a JAR file along with the Jenkins instance.

SSH

• On the other hand, the SSH service is initially disabled in Jenkins. To enable it, we need to ask Jenkins to pick a random port using the following command.

  Randomly assigned SSH port:

    $ curl -Lv https://JENKINS_URL/login 2>&1 | grep -i ‘x-ssh-endpoint’ 
    < X-SSH-Endpoint: localhost:53801     // Output - Returns randomly assign SSH port
  

  Now need to set up authentication for SSH. Jenkins relies on SSH-based public-private key authentication. To add an SSH public key for the relevant user, we need to navigate to the Jenkins configuration page and paste the public key there. After configuring authentication, we will have access to several built-in CLI commands in that specific Jenkins environment.

    $ ssh -l username -p 53801 localhost help
  

  We can execute the CLI help command by providing the username and the port, and it’s going to give us a full list of commands available in a given Jenkins environment. This command includes a lot of actions like building jobs, listing jobs, installing plugins, deleting the plugins, and a lot more.

  

  The above building jobs command allows users to trigger any job or pipeline for which they have permissions.

Jenkins CLI - JAR

While the SSH-based CLI is efficient and meets most of the requirements, there are scenarios where the Jenkins CLI client distributed with Jenkins is a more suitable choice.

• The default transport for the CLI client is HTTP, which eliminates the need to open additional ports in a firewall. IT is a Java application provided as a downloadable JAR file

We can use CLI to perform various actions including listing the jobs -

Building the jobs -

Or even getting the job XML definition file -

Jenkins REST API

Jenkins REST API provides a comprehensive set of endpoints for programmatic interactions with Jenkins. This allows developers to write scripts or applications that can control Jenkins remotely.

Actions: The API empowers you to perform a wider range of tasks, including: Creating, updating, and deleting jobs Triggering builds Retrieving job details, logs, and artifacts.

Managing nodes and agents Usage: We can access the API endpoints through HTTP requests using tools like curl or by utilising libraries in various programming languages (Python, Java, etc.). These libraries simplify API interactions and offer a more structured approach.

Authentication: Similar to the CLI, the REST API also supports authentication methods for secure access.

Basic Authentication: Pass your username and password using HTTP headers for basic authentication. This method is similar to the CLI's approach, but be cautious about storing credentials in code.

API Tokens: The recommended approach is to use API tokens generated within Jenkins. Include the token in the HTTP header (Authorization: Bearer ) for secure communication.

Building a job using Jenkins-CLI

Need to download Jenkins-CLI JAR file.

• Go to manage Jenkins → Jenkins CLI

• (After copying the JAR address) - wget <address>,

  example: wget http://localhost:8080/jnlpJars/jenkins-cli.jar

• java -jar jenkins-cli.jar -s http://localhost:8080/ help

There is no authentication done. Need to authenticate.

Now I am authenticated.

Now we have built a job using Jenkins CLI.

How REST API can be used

Jenkins provides a machine consumable API to its functionalities, and it comes with a couple of flavours like XML, JSON, and Python. We can use this API to retrieve information from Jenkins for programmatic consumption, triggering a new build, and creating jobs.

Accessing Jenkins through REST API. The link can be found from REST API section in Jenkins Dashboard (on the downside).

Quering for existing jobs using jq, because we have used json in place of xml in the url.

Checking a particular job using the URL -

Building a Job

Crumb is a security token used to cross-site request forgery, basically CSRF attacks in the browser. And it is a dynamic value, which changes frequently. And when making a REST API call to Jenkins, it is ideal to include this crumb in the request header to authenticate the request. So that’s one way of bypassing this error. So basically, you know, this error occures if the API call which we made to the Jenkins API is missing the crum.

We can use the crum or we can use the token.

Jenkins User → Configure / Security → API Token → Generate

Building the job

Installing a plugin using REST API

How to generate and use a CSRF crumb token

It is Cross-Site Request Forgery, which is a type of security vulnerability in web appliation. And to secure agains the CSRF attacts, Jenkins employs a token system called a crumb, which is a unique randomly generaed token that is associated with each user session.

It is enabled in Jenkins by default.

And we can disable this by setting the system property.

Now generate and use CSRF - CRUMB

So, This is the crumb tooken.

Request header and response header

Now we have the crumb and the cookie.

curl -s -u arindam:Pa55w0rD! \
  -c /tmp/cookies \
  http://localhost:8080/crumbIssuer/api/json

curl -s -u arindam:Pa55w0rD! \
  --cookie /tmp/cookies \
  -H "Jenkins-Crumb:Your-crumb-value" \
  -X POST http://localhost:8080/job/Demo2/build

Jenkins Security

Core principle of Jenkins security is the concept of least privilege. This means giving users only the permissions they absolutely need to perform their task within Jenkins.

• Authentication is the process of verifying somone’s identity. Required to provide credentials like username, password, API token, and the system checks if they are valid and belong to a real user.

• Authorisation is the process of determining what a user is allow to do after they have been authenticated.

Based on the identity and assigned permission, the system determines what action we can take, such as read, write, delete, etc,.

Jenkins offers several ways to manage who can access it, providing different levels of flexibility depending on your needs. Here is the breakdown of the built-in choices.

Built-in choices

• We have the Jenkins user database, which is a default option suitable for smaller setups, where we can manage user directly within Jenkins.

• We can use Unix user groups or Group databases. If your system uses UNIX-based user accounts, we can leverage them for Jenkins access control.

• Servlet container security integrates with the web services security model for user authentication.

• And finally, external LDAP can be used for larger organisations. Jenkins can connect to existing directory service like LDAP, allowing user to login with their usual corporate credentials.

Jenkins plugins

Jenkins plugins can further extend the security options.

• Some populer choices include Active Directoryto intigrate with Microsoft’s Active Directory for centralize user management.

• Make user of SAML 2.0 single sign-on to enable users to log into Jenkins using a the existing SSO credentials for a seamless experience.

Authentication in Jenkins - Choosing the right approach

The default internal database works well for smaller groups. However, for enterprice environments, using a corporate directory service like LDAP or Active Directory is recommended. This simplifies the user management and avoids the needs for separate credentials within Jenkins.

Authorization in Jenkins

It determines, what action user can perform.

• Resource represent the what, the task, object, or action a user might want to manipulate. It can be building a job or deleting a credential.

• Role represents collection of releted permissions grouped together by function.

• Requester represent who is the user or group trying to access a resource, tripically assigned one or more roles.

Matrix based authorization options:

• Matrix-based security

• Project-based matrix authorisation strategy

Both strategies look like a spreadsheet with permissions group by resources across the top and users or group names down the left side.

• Talking about matrix based security, which is a similar approach where users and groups are granted permissions globally across all the projects within Jenkins.

• Projec based matrix authorization strategy offrs more granular controlby allowing assign permissions based on the specific project a user or group access.

Always required to grant the least necessery access. This avoids creating bottlenecks in the CI/CD pipelines while maintaining good security.

Jenkins Authentication

Normally we can’s see the register option for another user. It would be better if we can configure Jenkins with the company’s LDAP server or Active Directory.

Currently, Jenkins using “Jenkin’s own user database”.

This is where Jenkins manage its own database. We can see the users here. Currently we have a single user.

We have disabled ‘Keep me sign in’ option and ‘allow user to sign up’ option.

Now we can see there is register option.

And now we have another user, that we just register.

Jenkins Authorization

By setting this option any unknown user can do anything with our jenkins. This is a security issue, so need never allow this option.

Now Anonymous users only have read access. We can varify, by seeing, there is no new job creation and manage Jenkins option.

This is a dummy security realm with no actual security. Feels like LDAP.

After installing the ‘Mock security plugin’ now we can set user with their groups. Now with these setting only configurede user can login to Jenkins.

Authorization

Installing Matrix authorization strategy plugin. And after installing the plugin we have two option in the authorization section (Dashboard → Manage Jenkins → Security)

We can add users or groups in the Matrix and configure authorisation for them.

Practice: Jenkins Pipeline and Basic Security

Q1. Jenkins is already installed on this machine, but we are unsure about its current status.

What is its status?

systemctl status jenkins

Q2. Create a new Pipeline Project in Jenkins with the following details:

• Project/Job Name: lab2-java-hello-world

• Add a description: Exploring Pipeline Projects

• Pipeline Definition: Pipeline Script

• Script: Use the sample Hello World Pipeline Script that is pre-provided by Jenkins

• Enable Use Groovy Sandbox

Apply the changes and build the job manually. Check out the logs and make sure the job finishes successfully.

You can access Jenkins by clicking the Jenkins icon on the top-right of the lab and using the below credentials (Hypothetical):

• username: admin

• password: Adm!n321

Answer:

1. Navigate to the Jenkins UI and use the credentials provided in the task statement to log in.

2. On the home page, click on Create a job.

3. Enter lab2-java-hello-world for Enter an item name, select Pipeline and click on OK.

4. In the General section of the job, enter Exploring Pipeline Projects under Description.

5. Scroll down to the Pipeline section, and select Pipeline Script under Definition.

6. On the top right of the Script prompt, click on the small dropdown that says try sample Pipeline and select Hello World here.
   You should see the prompt get auto-filled with the sample job definition.

7. Check the Use Groovy Sandbox option below the Script prompt. Click on Apply and then Save.

Your job will be created now, and the job dashboard will be visible. On the left, click on Build Now. Then under Build History, click on your build number.

Now, on the left panel, click on Console Output and you should see that your job has finished successfully.

Q3. A Gitea Server is created and used as an SCM throughout these labs.

Within Gitea, we will be using the dasher-org organization. Within this organization, we have a repository named jenkins-hello-world. Go through the repo. It is a simple Java Springboot application with a /hello endpoint and a few test cases.

Create a file named Jenkinsfile in the main branch of the repo:

• Within Jenkinsfile, copy and paste the skeleton-pipeline script from the ~/lab-data/lab2-q3-pipeline.txt file.

• Within the tools section, configure maven as maven-398.

• In the Build stage:

  • Add a git cmd to Checkout SCM.

  • Use the pipeline syntax generator to generate the pipeline script if required.

  • Repo URL: http://localhost:5555/dasher-org/jenkins-hello-world.git

  • Branch: main

  • Add a Maven cmd to build the application and skip tests:

mvn clean package -DskipTests=true

• Add a new stage named Unit Test. Add a single step that executes the below command:

mvn test

Commit the changes to the repo.

You can access Gitea by clicking the Gitea icon on the top right and logging in using the credentials below:

• username: gitea-admin

• password: gitea-password

Note: We have already cloned the jenkins-hello-world repo in the terminal under /root.

Answer:

1. Navigate to /root/jenkins-hello-world repo. You should be on branch main now.

2. Run the following command to copy the skeleton file provided in the repo:

cp /root/lab-data/lab2-q3-pipeline.txt .

Then rename this file:

mv lab2-q3-pipeline.txt Jenkinsfile

3. Now, edit the Jenkinsfile so that the final product is as shown:

pipeline {
    agent any

    tools {
        // Install the Maven version configured as "maven-398" and add it to the path.
        maven "maven-398"
    }

    stages {
        stage('Check Maven') {
            steps {
                sh "if ! command -v mvn; then echo 'Maven not found!'; exit 1; fi"
            }
        }

        stage('Build') {
            steps {
                git branch: 'main', url: 'http://localhost:5555/dasher-org/jenkins-hello-world.git'
                sh "mvn clean package -DskipTests=true"
            }
        }

        stage('Unit Test') {
            steps {
                sh "mvn test"
            }
        }
    }
}

4. Then run the git status command and you should see that the Jenkinsfile is waiting to be committed.

5. Run the following commands to commit the file to the repo:

git add .
git commit -m "Added Jenkinsfile"
git push origin main

Q4. Within the Jenkins UI, switch to the lab2-java-hello-world project.

Modify the pipeline definition to Pipeline script from SCM:

• SCM: Choose Git (Install the plugin if not already

• Repository URL: - http://localhost:5555/dasher-org/jenkins-hello-world.git

• Branch: */main

• Script Path: Jenkinsfile

Apply the changes and build the job manually.

Note: The build should fail, check out the logs.

Jenkins Credentials

• username: admin

• password: Adm!n321

Answer:

1. Go to the lab2-java-hello-world job dashboard from the Jenkins UI.

2. Click on Configure on the left panel.

3. Scroll down to the Pipeline section and select Pipeline script from SCM under Definition.

4. Now, under SCM, select Git from the dropdown and add http://localhost:5555/dasher-org/jenkins-hello-world.git under Repository URL.

5. You should see */master pre-filled for Branch Specifier under Branches to build. Change this to */main. You should also see Jenkinsfile under Script Path at the bottom of the page. If not, then enter this value.

6. Click on Apply and Save.

7. Now, click on Build from the left panel. Your job should get built and fail.

Q5. The previous job build resulted in a failure because of the below error:

Tool type "maven" does not have an install of "maven-398" configured - did you mean "null"? @ line 5, column 15.
           maven "maven-398"
                 ^

Your task is to install Maven and set its path with the below config:

• Maven version: 3.9.8

• Config Name: maven-398

• Install automatically from Apache

Now, go back to lab2-java-hello-world project in Jenkins and re-build it.

The build should fail. Check out the logs and select which stage fails.

Jenkins Credentials

• username: admin

• password: Adm!n321

Answer:

1. Click on Manage Jenkins from the Jenkins dashboard.

2. Scroll down to System Configuration and click on Tools.

3. Scroll down to Maven installations and click on Add Maven.

4. Under Name, enter maven-398. Check Install automatically, and, under Install from Apache, select version 3.9.8.

5. Click on Apply and then Save.

6. Go to the lab2-java-hello-world job dashboard and click on Build Now. Wait for some time for the build to complete and you should see it fail.

7. Click on your build number, and then on Pipeline Overview.

You should see here that the Unit Testing stage failed.

Q5. In this task, you will fix the Unit Test error and archive JUnit-formatted test results.

Fixing Unit Test error
Within jenkins-hello-world repository, modify below data:

• File name: HelloControllerTests.java

• Path: src/test/java/com/kodekloud/hello_demo/HelloControllerTests.java

• On line 63, replace Hola with Hello.

• Commit the changes.

Archiving JUnit-formatted test result
Within the same repo, modify the Jenkinsfile with below data:

• Within the Test stage after the maven test command, add another command to archive JUnit-formatted test result.

• Test result path: target/surefire-reports/TEST-*.xml

• Check plugins and install the Junit plugin that is not yet installed.

Finally, execute the build job.

Jenkins credentials

• username: admin

• password: Adm!n321

Answer:

Steps for fixing the Unit Test error

1. Navigate to the jenkins-hello-world repository from the terminal.

2. Go to the following path: src/test/java/com/kodekloud/hello_demo:

cd src/test/java/com/kodekloud/hello_demo

You should see the required file here:

root@jenkins-server com/kodekloud/hello_demo on  main ➜  ls
HelloControllerTests.java

3. Open the file using the vi editor:

vi HelloControllerTests.java

Remain in the normal mode, type :63 and click on Enter.
The cursor will be placed on the following line:

.andExpect(content().string(startsWith("Hola")));

Now, change Hola to Hello by switching to insert mode and save the file thereafter.

4. Run the following commands to commit your changes:

git add .
git commit -m "Modified HelloControllerTests java file"
git push origin main

Steps for archiving JUnit-formatted test result

1. First, install the JUnit plugin. From the Jenkins dashboard, go to Manage Jenkins->Plugins->Available Plugins and search for JUnit.

2. Click on Install and then restart Jenkins.

3. Now, go to Pipeline Syntax->Snippet Generator on the job dashboard, and, under Sample Step, select junit: Archive JUnit-formatted test results. Then enter target/surefire-reports/TEST-*.xml under Test report XMLs and click on Generate Pipeline Script button at the bottom of the page.
   You should see an output like this:

junit 'target/surefire-reports/TEST-*.xml'

NOTE: The JUnit syntax may vary, but it will archive the tests. Copy it.

4. Now, in your terminal, again open the Jenkinsfile again for editing in the jenkins-hello-world repository. To the Unit Test stage, add the command that we copied from the pipeline syntax generator:

stage('Unit Test') {
            steps {
                sh "mvn test"
                junit stdioRetention: '', testResults: 'target/surefire-reports/TEST-*.xml'
            }
        }

Save the file and commit the changes:

git add .
git commit -m "Modified Jenkinsfile file"
git push origin main

5. Finally, execute the build job and wait for it to complete successfully.

Q7. The Jenkinsfile within jenkins-hello-world repo has been updated. Please have a look.

In this task, we will make use of the Build Parameters to replace the Application Port and Sleep time within the Integration testing stage.

Go back to the lab2-java-hello-world project in Jenkins and parameterize it with the below data:

• Add a string parameter:

  • Name: APPLICATION_PORT

  • Default Value: 6767

  • Description: Application port on which integration tests should happen

• Add a choice parameter:

  • Name: SLEEP_TIMER

  • Choices:
    5s
    10s
    20s
    40s
    60s

  • Description: Time to sleep before initiating integration tests

Update the Jenkinsfile within jenkins-hello-world repo to replace the sleep time and port number in the Integration testing stage with the above parameters.

Commit the changes to the repository. Then, manually build the lab2-java-hello-world project with parameters.

The build should pass. Check out the logs.

Jenkins Credentials
URL: http://localhost:8085
Username: admin
Password: Adm!n321

Answer:

Adding parameters

1. Navigate to the lab2-java-hello-world project from the Jenkins UI.

2. Click on the Configure option in the left-hand menu of the lab2-java-hello-world project.

3. Scroll down to the "General" section. Check the box labeled "This project is parameterized" and click the "Add Parameter" button.

4. From the dropdown list, select "String Parameter". Configure the parameter as follows:
   Name: APPLICATION_PORT
   Default Value: 6767
   Description: Application port on which integration tests should happen

5. Click the "Add Parameter" button again. From the dropdown list, select "Choice Parameter." Configure the parameter as follows:
   Name: SLEEP_TIMER
   Choices:

5s
10s
20s
40s
60s

Description: Time to sleep before initiating integration tests

Updating Jenkinsfile

1. Navigate to the jenkins-hello-world repository in your terminal.

2. Edit the Jenkinsfile. Modify the integration testing step as shown below:

stage('Integration Testing') {
    steps {
        sh "sleep ${params.SLEEP_TIMER}"
        sh """ curl -s http://localhost:${params.APPLICATION_PORT}/hello | grep -i "Hello, KodeKloud community!" """
    }
}

3. Save the file and commit the changes:

git add .
git commit -m "Added params in integration testing step"
git push origin main

Now, go to the job dashboard in Jenkins and click on Build Now. You will get APPLICATION_PORT and SLEEP_TIMER variable values pre-filled. Click on the green Build button at the bottom.

Q8. In this task, we will work with the Jenkins CLI.

Download Jenkins CLI JAR to the root directory by the name jenkins-cli.jar.

Then, using the CLI, install the Blue Ocean plugin with version 1.27.14. The plugin should be activated right away without postponing until the reboot.

Jenkins Credentials
URL: http://localhost:8085
Username: admin
Password: Adm!n321

Answer:

To download the Jenkins CLI jar file:

1. In the Jenkins UI, go to Manage Jenkins -> Tools and Actions -> Jenkins CLI.

2. You should see this line at the top of the page:

You can access various features in Jenkins through a command-line tool. See the documentation for more details of this feature. To get started, download jenkins-cli.jar, and run it as follows:

Right click on jenkins-cli.jar and copy the link address.

3. Then in your terminal, navigate to the /root directory and run the following command:

curl https://8085-port-605ee7c8cad74a3a.kk-lab-dev.kodekloud.com/jnlpJars/jenkins-cli.jar -o jenkins-cli.jar

You might have a different URL than this one.
You should see a new file jenkins-cli.jar in your terminal now.

To install the Blue Ocean plugin using the newly downloaded Jenkins CLI JAR, run the following command in your terminal:

java -jar jenkins-cli.jar -s http://localhost:8085/ -auth 'admin:Adm!n321' install-plugin blueocean:1.27.14 -deploy

Q9. Use Jenkins REST API to build a job with the following parameters:

• Job Name: lab2-java-hello-world

• Parameter:

  • Name: SLEEP_TIMER

  • Value: 20s

• Authentication Credentials:

  • Username: admin

  • Password: <api-token>

  • Generate an API token for the Jenkins user. The name of the token should be: rest-api-token.

• Use curl to run the REST API, and, if build is triggered, copy the entire curl command to /root/solutions/task9.txt file.

Jenkins Credentials
Username: admin
Password: Adm!n321

Answer:

To generate the API token, go to admin->Security->API Token. Here, click on the Add new token button.

2. Enter rest-api-token in the Default name blank and click on Generate.

3. Immediately copy the token and store it securely. Scroll down in the Jenkins UI and click on Save.

4. Now, use this token to create your job using the curl command:

curl -u admin:116c5ea72d32ef0c1fc0106a968738d606 -X POST http://localhost:8085/job/lab2-java-hello-world/buildWithParameters -d SLEEP_TIMER=20s

Here, admin is your username and the token is placed after the colon :.

5. Go to your Jenkins UI and inspect the lab2-java-hello-world job. You should see a build running. Click on that build number and go to Parameters from the menu on the left sidebar.
   You should see the value of SLEEP_TIMER as 20s, confirming that our curl command worked.

Now, copy the command that you ran to the file /root/solutions/task9.txt.

Q10. We have created a new user in Jenkins. Go ahead and check out the user details in the Jenkins UI.

The user credentials are as follows:

• Username: zack

• Password: password123

Using Jenkins CLI with Zack's credentials, run the below commands:

• Dumps the job definition XML to stdout

java -jar jenkins-cli.jar -s http://<jenkins-url>/ -auth <zack-username:zack-password> get-job lab2-java-hello-world

• Builds a job

java -jar jenkins-cli.jar -s http://<jenkins-url>/ -auth <zack-username:zack-password> build lab2-java-hello-world -f

Notice that both the above commands get executed successfully.

As a Jenkins Admin, you need to ensure that Zack can only access job definitions and restrict his ability to build jobs.

The Matrix Authorization Strategy plugin is already installed on Jenkins. Make changes within the Authorization settings to restrict Zack’s ability to build jobs.

Re-run the Builds a job command:

java -jar jenkins-cli.jar -s http://<jenkins-url>/ -auth <zack-username:zack-password> build lab2-java-hello-world -f

This should then result in:

ERROR: zack is missing the Job/Build permission

Jenkins URL: http://localhost:8085/
Username: admin
Password: Adm!n321

Answer:

Run the following command in the terminal:

java -jar jenkins-cli.jar -s http://localhost:8085/ -auth zack:password123 get-job lab2-java-hello-world

You should see the job definition XML as the output.

Also, run this command:

java -jar jenkins-cli.jar -s http://localhost:8085/ -auth zack:password123 build lab2-java-hello-world -f

Go to the job dashboard in the Jenkins UI and you should see a new build triggered.

We need to restrict this ability of building jobs for Zack.
To do this, follow the below steps:

1. Go to Manage Jenkins->Security.

2. Under Authorization, select Matrix-based security from the dropdown. You should see a grid now with various permissions.

3. Click on the Add user button at the bottom and enter zack in the User ID prompt that appears. Click on OK.

4. Select the following permissions (checkboxes) for Zack:

   • Read in the overall section.

   • Read in the Job section.

5. Click on the Save button at the bottom of the page.

Now, run the build job command provided above again. You should see an output like this:

ERROR: zack is missing the Job/Build permission

This confirms that we have restricted access for Zack.