010 - Jenkins for Beginners: Your First Steps in CI/CD Automation

Welcome to our DevOps series! Today, we dive into one of the cornerstones of Continuous Integration and Continuous Delivery (CI/CD): Jenkins. Jenkins is a powerful, open-source automation server that helps automate the parts of software development related to building, testing, and deploying, facilitating CI/CD pipelines.

This article will guide you through setting up Jenkins, configuring essential tools, and creating different types of jobs to automate the build process for a sample Maven project.

Prerequisites:

• A server (Linux recommended, like Ubuntu) with Docker installed.

• Basic understanding of Linux commands.

• Git installed on your local machine and ideally on the server.

• A sample Java Maven project hosted in a Git repository (e.g., on GitLab). Create a simple one if you don't have one.

• Basic familiarity with Maven concepts (pom.xml, goals like clean, install, package).

Let's get started!

1. Setting Up Jenkins using Docker

Using Docker is one of the easiest ways to get Jenkins up and running quickly.

Steps:

1. Create a Docker Volume for Persistence: This ensures your Jenkins configuration and job data persist even if the container stops or is removed.

    docker volume create jenkins_home
   

2. Run the Jenkins LTS Docker Image: We'll use the official Long-Term Support (LTS) image.

    docker run -d \
      -p 8080:8080 \
      -p 50000:50000 \
      --name jenkins-server \
      --restart=on-failure \
      -v jenkins_home:/var/jenkins_home \
      jenkins/jenkins:lts-jdk17
   

   • -d: Run in detached mode (background).

   • -p 8080:8080: Map port 8080 on your host to port 8080 in the container (for the Jenkins web UI).

   • -p 50000:50000: Map port 50000 for communication with Jenkins agents (optional for this basic setup, but good practice).

   • --name jenkins-server: Assign a name to the container.

   • --restart=on-failure: Automatically restart the container if it crashes.

   • -v jenkins_home:/var/jenkins_home: Mount the Docker volume we created into the container's Jenkins home directory.

   • jenkins/jenkins:lts-jdk17: The official Jenkins LTS image with JDK 17.

3. Access Jenkins & Initial Setup:

   • Open your web browser and navigate to http://<your_server_ip>:8080.

   • Jenkins needs an initial administrator password to unlock. Find it by running:

       docker exec jenkins-server cat /var/jenkins_home/secrets/initialAdminPassword
     

   • Copy the password, paste it into the Jenkins setup screen, and click "Continue".

   • Choose "Install suggested plugins". Jenkins will install a standard set of useful plugins.

   • Create your first admin user account and click "Save and Continue".

   • Confirm the Jenkins URL (it should be correct) and click "Save and Finish".

   • Click "Start using Jenkins". You are now logged into your Jenkins dashboard!

2. Setting Up Build Tools (Maven) via Dashboard

Jenkins needs to know where build tools like Maven are located.

Steps:

1. Go to Manage Jenkins -> Global Tool Configuration.

2. Scroll down to the Maven section and click Add Maven.

3. Give it a Name (e.g., Maven-3.8.6).

4. Check the Install automatically box.

5. Select a recent version from the Install from Apache dropdown (e.g., 3.8.6).

6. Click Save. Jenkins will now automatically download and install this Maven version when a job requires it.

3. Saving Credentials in Jenkins

Never hardcode passwords or API keys in your job configurations or scripts! Jenkins provides a secure Credentials Manager. Let's add credentials for accessing a private Git repository.

Steps:

1. Go to Manage Jenkins -> Manage Credentials.

2. Under Stores scoped to Jenkins, click on the (global) domain.

3. Click Add Credentials (usually on the left menu).

4. Select the Kind: Username with password.

5. Enter Scope as Global.

6. Enter the Username for your Git provider (e.g., your GitHub username).

7. Enter the Password (for GitHub, this is often a Personal Access Token (PAT) rather than your actual password).

8. Give it a descriptive ID (e.g., github-credentials). This ID is used to refer to these credentials in jobs.

9. Add an optional Description.

10. Click Create.

4. Setting up Docker Integration (Sharing Host Docker Socket)

If your Jenkins jobs need to build or run Docker containers, the easiest way is often to let the Jenkins container use the Docker daemon running on the host server.

Important Security Note: This method grants the Jenkins container significant control over the host's Docker environment. Be aware of the security implications.

Steps:

1. Stop and Remove the existing Jenkins container:

    docker stop jenkins-server
    docker rm jenkins-server
   

   (Don't worry, your data is safe in the jenkins_home volume)

2. Rerun the Jenkins container, adding the Docker socket volume mount:

    docker run -d \
      -p 8080:8080 \
      -p 50000:50000 \
      --name jenkins-server \
      --restart=on-failure \
      -v jenkins_home:/var/jenkins_home \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v $(which docker):/usr/bin/docker \
      jenkins/jenkins:lts-jdk17
   

   • -v /var/run/docker.sock:/var/run/docker.sock: This mounts the host's Docker socket into the container, allowing the container to communicate with the host's Docker daemon.

   • -v $(which docker):/usr/bin/docker: This mounts the host's Docker binary into the container (useful so Jenkins can find and execute the docker command). Note: This path might vary slightly depending on your host OS setup.

3. Install Docker CLI inside the container (if needed): Sometimes the base Jenkins image doesn't include the Docker CLI client, even if it can talk to the socket. You might need to install it within the container or use a Jenkins image that includes it. For simple cases, mounting the binary as above often suffices. If you encounter issues, you might need to:

   • Exec into the container: docker exec -it -u root jenkins-server bash

   • Install Docker CLI: (Commands vary based on the container's base OS, e.g., apt-get update && apt-get install -y docker.io for Debian/Ubuntu based).

   • Exit the container: exit

Alternative: Consider using official Jenkins images with Docker pre-installed, like jenkins/docker.

4. Grant Permissions (If Necessary): The user running Jenkins inside the container (often jenkins) needs permission to access the Docker socket. One way is to add the jenkins user to the docker group inside the container. The group ID (gid) of the docker group on the host must match the gid used inside the container. This can be complex. A simpler (but less secure for the host) approach for testing is to ensure the jenkins user inside the container runs with sufficient privileges or match the host's docker group GID when starting the container using the -u and --group-add flags. For this basic guide, we'll rely on the Jenkins process often having enough rights initially or the user manually handling permissions inside the container using docker exec -it -u root jenkins-server bash and chmod 666 /var/run/docker.sock if needed.

5. Setting Up a Freestyle Job

Freestyle jobs are configured entirely through the Jenkins web UI. They are good for simple tasks.

Goal: Check out code from Git and build it using Maven.

Steps:

1. On the Jenkins dashboard, click New Item.

2. Enter an item name (e.g., My-Maven-Freestyle-Job).

3. Select Freestyle project and click OK.

4. Description: (Optional) Add a description of the job.

5. Source Code Management:

   • Select Git.

   • Enter the Repository URL of your sample Maven project (e.g., https://github.com/your-username/your-maven-repo.git).

   • Select the Credentials you created earlier (github-credentials) from the dropdown. If your repo is public, you might not need credentials.

   • Specify the Branch to build (e.g., */main or */master).

6. Build Triggers: (Optional) You can configure triggers like Poll SCM (check Git periodically) or Build periodically. For now, we'll trigger manually.

7. Build Environment: (Optional) Configure settings like deleting the workspace before build.

8. Build:

   • Click Add build step.

   • Select Invoke top-level Maven targets.

   • Choose the Maven Version you configured (Maven-3.8.6).

   • In Goals, enter clean install. This tells Maven to clean the project and then compile, test, and package it.

9. Post-build Actions: (Optional) Add steps like archiving artifacts or sending notifications.

10. Click Save.

Run the Job: Go to the job's page and click Build Now on the left menu. You can watch the progress in the Build History and view the detailed Console Output.

6. Setting Up a Pipeline with SCM Script

Jenkins Pipeline allows you to define your entire build process as code using a Groovy-based DSL, typically stored in a file named Jenkinsfile within your project's Git repository. This is the modern, recommended approach (Pipeline as Code).

Goal: Define the same Maven build process as above, but using a Jenkinsfile.

Steps:

1. Create a Jenkinsfile: In the root directory of your sample Maven project Git repository, create a file named Jenkinsfile (case-sensitive, no extension) with the following content:

    // Jenkinsfile (Declarative Pipeline)
    pipeline {
        agent any // Run on any available Jenkins agent/node
   
        tools {
            maven 'Maven-3.8.6' // Use the Maven tool configured in Jenkins
        }
   
        stages {
            stage('Checkout') {
                steps {
                    // Automatically checks out code from the SCM configured in the job
                    checkout scm
                }
            }
            stage('Build') {
                steps {
                    // Execute Maven command
                    sh 'mvn clean install'
                }
            }
            stage('Example Docker Step (Optional)') {
                // Only runs if Docker is configured correctly (Step 4)
                steps {
                    script {
                        // Example: Show docker version using the host's docker
                        sh 'docker --version'
                        // Example: Build a docker image if you have a Dockerfile
                        // docker.build("my-app:${env.BUILD_NUMBER}")
                    }
                }
            }
        }
   
        post {
            always {
                echo 'Pipeline finished.'
                // Clean up workspace after build
                cleanWs()
            }
            success {
                echo 'Pipeline succeeded!'
                // Archive build artifacts (e.g., the JAR/WAR file)
                archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
            }
            failure {
                echo 'Pipeline failed!'
                // Send notifications, etc.
            }
        }
    }
   

2. Commit and Push Jenkinsfile: Add Jenkinsfile to your Git repository and push it.

    git add Jenkinsfile
    git commit -m "Add Jenkinsfile for CI pipeline"
    git push origin main # Or your branch name
   

3. Create the Pipeline Job in Jenkins:

   • On the Jenkins dashboard, click New Item.

   • Enter an item name (e.g., My-Maven-Pipeline-SCM).

   • Select Pipeline and click OK.

   • Description: (Optional) Add a description.

   • Scroll down to the Pipeline section.

   • Select Definition: Pipeline script from SCM.

   • Select SCM: Git.

   • Enter the Repository URL of your project.

   • Select the Credentials if needed.

   • Specify the Branch to build (e.g., */main).

   • Script Path: Keep the default Jenkinsfile. This tells Jenkins to look for a file named Jenkinsfile in the root of the repository.

   • Check Lightweight checkout if available (usually default and preferred).

   • Click Save.

4. Run the Job: Go to the job's page and click Build Now. Jenkins will check out the code (including the Jenkinsfile) and execute the stages defined within it. You'll see a visual representation of the stages in the Jenkins UI.

7. Setting Up a Multibranch Pipeline

Multibranch Pipelines are powerful for projects with multiple branches (like feature branches, release branches). Jenkins automatically discovers branches and pull requests containing a Jenkinsfile and creates corresponding Pipeline jobs.

Goal: Automatically build any branch in your repository that contains a Jenkinsfile.

Steps:

1. Ensure Jenkinsfile exists: Make sure the Jenkinsfile created in Step 6 is present in the branches you want to build (e.g., main, develop, feature branches).

2. Create the Multibranch Pipeline Job:

   • On the Jenkins dashboard, click New Item.

   • Enter an item name (e.g., My-Maven-Multibranch-Project).

   • Select Multibranch Pipeline and click OK.

   • Display Name & Description: (Optional).

   • Under Branch Sources, click Add source.

   • Select Git.

   • Enter the Project Repository URL.

   • Select Credentials if needed.

   • Under Behaviors: Jenkins usually auto-detects branches with a Jenkinsfile. You can add behaviors to filter branches, discover pull requests, etc. The defaults are often sufficient to start.

   • Build Configuration: Ensure Mode is set to by Jenkinsfile. The Script Path should be Jenkinsfile.

   • Scan Multibranch Pipeline Triggers: Configure how often Jenkins scans the repository for new/deleted branches (e.g., Periodically if not otherwise run). Set an interval like 1 minute for testing or 1 hour for production.

   • Click Save.

Run/Scan: Jenkins will automatically scan the repository. Go to the My-Maven-Multibranch-Project job page. You should see Jenkins discovering branches that contain a Jenkinsfile and creating sub-jobs for them (e.g., My-Maven-Multibranch-Project/main). It will trigger builds for these branches automatically based on the Jenkinsfile. Subsequent pushes to these branches (or new branches with a Jenkinsfile) will trigger new scans and builds according to your trigger configuration.

8. Auto-Incrementing Version using Script (Maven Example)

Often, you want each build to have a unique version, perhaps incorporating the Jenkins build number. We can modify the pom.xml version during the build using Maven commands within our Jenkinsfile.

Goal: Set the Maven project version to something like 1.0.${BUILD_NUMBER}-SNAPSHOT during the build.

Modify Jenkinsfile:

Update the Build stage in your Jenkinsfile:

// Jenkinsfile (Declarative Pipeline) - Snippet Update
pipeline {
    agent any

    tools {
        maven 'Maven-3.8.6'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        stage('Set Version & Build') {
            steps {
                script {
                    // Construct the new version using Jenkins environment variable BUILD_NUMBER
                    def newVersion = "1.0.${env.BUILD_NUMBER}-SNAPSHOT"
                    echo "Setting project version to: ${newVersion}"

                    // Use Maven Versions Plugin to set the version in pom.xml
                    // 'versions:set' modifies the pom.xml in the workspace
                    // '-DnewVersion=...' passes the version variable
                    // '-DgenerateBackupPoms=false' avoids creating backup pom files
                    sh "mvn versions:set -DnewVersion=${newVersion} -DgenerateBackupPoms=false"

                    // Now run the build with the updated pom.xml
                    echo "Building project with version ${newVersion}"
                    sh 'mvn clean package' // Using package goal to create the artifact
                }
            }
        }
        // Add other stages like Test, Deploy etc.
    }

    post {
         always {
            echo 'Pipeline finished.'
            // Commit the modified pom.xml back? (Requires Git credentials and careful handling)
            // For simplicity, we are NOT committing the version change back to Git here.
            // The version change only exists during this build execution.

            // Clean up workspace (removes modified pom.xml)
             cleanWs()
         }
        success {
            echo 'Pipeline succeeded!'
            // Archive the artifact. Its name might now include the build number if
            // your pom.xml uses ${project.version} in the <finalName> tag.
            archiveArtifacts artifacts: "target/*-${newVersion}.jar", fingerprint: true // Adjust artifact name pattern
        }
        // ... other post actions
    }
}

Explanation:

1. env.BUILD_NUMBER: Jenkins automatically provides environment variables, including the unique BUILD_NUMBER for each execution of a job.

2. def newVersion = ...: We define a Groovy variable newVersion combining a base version with the build number.

3. mvn versions:set ...: This command uses the Maven Versions Plugin (often included or easily added) to modify the <version> tag in your pom.xml file within the Jenkins workspace for this specific build.

4. -DgenerateBackupPoms=false: Prevents cluttering the workspace with pom.xml.versionsBackup files.

5. mvn clean package: We run the package goal (or install) after the version has been updated in the pom.xml. If your pom.xml uses ${project.version} in artifact naming, the resulting JAR/WAR will include the build number.

6. Important: This example does not commit the modified pom.xml back to your Git repository. The version change is ephemeral, existing only within the Jenkins workspace for the duration of the build. Automatically committing version bumps requires careful handling of credentials, potential merge conflicts, and deciding on your branching strategy. For many CI scenarios, using the build number in the artifact name (achieved by mvn versions:set followed by package) is sufficient without committing the change.

Essential Things for Beginners

• Workspace: Jenkins checks out code and performs builds in a dedicated directory called a workspace for each job.

• Plugins: Jenkins' power comes from its vast ecosystem of plugins. Explore Manage Jenkins -> Manage Plugins to find plugins for Git, Docker, notifications, testing frameworks, cloud providers, etc.

• Nodes/Agents: While we ran everything on the built-in node, real-world Jenkins setups use separate 'agent' machines (nodes) to distribute build load and handle different environments (e.g., Windows agent, Linux agent). Configure them under Manage Jenkins -> Manage Nodes and Clouds.

• Security: Always configure security (Manage Jenkins -> Configure Global Security). Enable authentication and authorization. Don't run Jenkins as root if avoidable. Use the Credentials Manager diligently.

• Console Output: This is your best friend for debugging. Check the detailed logs for any build to understand what happened.

Conclusion

You've successfully set up Jenkins using Docker, configured Maven, managed credentials, integrated Docker, and created Freestyle, Pipeline, and Multibranch Pipeline jobs! You also learned how to dynamically set the project version during a build.

I hope this article helps you understand the basics of Jenkins and its Jobs. This is just the beginning. Jenkins is incredibly flexible. Explore its features, browse available plugins, and start building more complex and robust CI/CD pipelines to automate your software development lifecycle. Feel free to reach out to me if you have any questions. Happy automating!

Summary

Welcome to our DevOps series on Jenkins, an open-source automation server that facilitates CI/CD pipelines. This guide covers setting up Jenkins using Docker, configuring Maven for builds, saving credentials securely, and integrating Docker. We'll explore creating Freestyle, Pipeline, and Multibranch Pipeline jobs, as well as auto-incrementing version numbers in Maven projects. This article aims to equip you with the basics of automating your software development lifecycle using Jenkins, setting you up for more complex CI/CD workflows.