Hello Every One in this blog I would like to explain about basics of what are Jenkins agents how to connect EC2 instance as a permanent agent to Jenkins controller running locally.

Agents

Jenkins agents (also referred to as nodes) are separate machines (physical or virtual) that run jobs assigned by the Jenkins master (controller). Master can share workload on these machines. By default Jenkins comes with single node where jobs are run on the same master node .

An agent can run on a different machine (remote or local), on a container, or in the cloud.

Use of Jenkins Agents

Agents help distribute the load of running jobs, which is especially useful when the master is under heavy load.
With agents, multiple jobs can run in parallel on different machines, which reduces build times.
Sometimes jobs need to run in specific environments (e.g., testing in both Linux and Windows). Agents allow you to run jobs in these various environments.

Type of Agents

Permanent Agents
Dynamic Agents

Permanent Agents

These are dedicated agents manually connected to Jenkins controller for specific jobs. These type of agents stay connected until they explicitly stopped.
Typically used for long-running, stable agents in environments that require specific configuration

Dynamic Agents

These agents are created on-demand when jobs are triggered and are terminated when the job is finished.

Example: AWS EC2 instances , Containers, Kubernetes pods.

How Jenkins Connects to Agents

SSH:

Jenkins uses Secure Shell (SSH) to connect to a remote agent. Jenkins pushes jobs over SSH to the agent machine.
JNLP (Java Network Launch Protocol):

Agents initiate the JNLP connection to the master. This is particularly useful if the agent is behind a firewall or NAT.

Setting EC2 instance as a permanent agent

After getting a basics of agents and how does it work. Lets set up a virtual machine as an agent to the Jenkins controller running locally.

On the local machine :

By clicking Build Executor Status we will be able to create nodes, select new node , select Permanent Agent as this is our agenda today.

executors refer to the number of concurrent jobs (or builds) that a Jenkins agent can run at the same time.

Suppose if we mention 4 executors. Jenkins can assign 4 jobs to run in parallel on that Linux machine. If you try to assign more than 4 jobs, the additional jobs will wait in the queue until one of the executors is free.

The Remote Root Directory in Jenkins refers to the directory on an agent machine where all the files related to the Jenkins jobs are stored.

Labels are used by the controller to detect on which agent jobs has to run.

Launch Method here we are going to look at

Launch Agent via SSH
Launch agents by connecting to controller.

Launch Agent via SSH

In this blog I will try to explain

Non Verifying Verification Strategy
Known Host files verification Strategy.

Under Host Key Verification Strategy.

SSH Host Key Verification: Normally, when we connect to a server via SSH, our SSH client checks if the server's SSH host key matches a known value. This helps us ensure that we are connecting to the right server and not an imposter.
Non-Verifying Strategy: When using the Non-Verifying Verification Strategy, Jenkins skips this verification step. It means Jenkins will connect to the agent without checking if the agent's SSH host key is valid or known.

Non Verifying Verification Strategy

Put the IP of instance in the host section.

In the Credentials Section we can select Credential Kind as Username and Password or Username Private Key

sudo passwd ec2-user we can set password for ec2-user. and we can use in credentials.

Now we can use these credentials.

We can even select Username and Private Key Option. Lets find how does this work.

Using ssh-keygen command on master node we get public and private keys.

Copy the Public Key: Take the file ~/.ssh/id_rsa.pub from the master machine and put its contents into the file ~/.ssh/authorized_keys on the agent machine.
Use the Private Key: On the master machine, use the private key file ~/.ssh/id_rsa to set up the connection in the credentials section of Jenkins.

By following any of the method master node could be able to connect to Agent node.

Known Host files verification Strategy.

Just simply follow above Username & private key method and add agents IP in known host file like this on master node.

ssh-keyscan -H agent-ip >> ~/.ssh/known_hosts

By this way agents would be able to connect vias SSH.

Launch agents by connecting to controller.

If we simply select this option Jenkins provides us commands to run on the agent node. Simply run those commands on agent node. Agent node is now authenticated and will be connected to master node.

But there is a catch here as our Jenkins master is running locally agent could not be able to find the master node. In such cases we have to use Reverse Proxy like Nginx, Ngrok, PageKite, serveo.net etc.. through this we can transfer traffic from these reverse proxies to our Local Jenkins Master node.

I have used Ngrok which is so simple to setup just navigate to ngrok official website they have provided clear guide to walk us through.

This Simple command generates a secure URL and http traffic which hits this URL will be transferred to http://localhost:8080.

ngrok http http://localhost:8080

Simple right.

Simple Pipeline Job using Jenkins agents.

pipeline {
    agent none  // Pipeline will not run on the master node

    stages {
        stage('Build and Test on All Platforms') {
            parallel {
                stage('Linux Build and Test') {
                    agent { label 'linux' }  // Assign the job to the Linux agent
                    steps {
                        // Commands to build and test on Linux
                        sh 'make build-linux'
                        sh 'make test-linux'
                    }
                }
                stage('Windows Build and Test') {
                    agent { label 'windows' }  // Assign the job to the Windows agent
                    steps {
                        // Commands to build and test on Windows
                        bat 'make build-windows'
                        bat 'make test-windows'
                    }
             }
    }
}

By using agents and the labels attached to them, we can specify on which node a pipeline job should run. Similarly, as in the example above, we can define which stage should run on a particular node..

Thanks for reading my blog. Have a great day🎉🎉🎉.

Jenkins Agents Part 1