How to run jenkins agents on kubernetes cluster
Anant Saraf13 min read
I already have a volume that has setup for jenkins and it is deployed on docker so i import it and use it
On docker desktop import this volume
Check the latest image
now run the jenkins docker image with latest version
docker run -d -p 8080:8080 -p 50000:50000 --name jenkins-2.452.2-lts -v jenkins-data:/var/jenkins_home jenkins/jenkins:2.452.2-lts
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
5a46ab7b7d2f jenkins/jenkins:2.452.2-lts "/usr/bin/tini -- /u…" 9 seconds ago Up 7 seconds 0.0.0.0:8080->8080
/tcp, 0.0.0.0:50000->50000/tcp jenkins-2.452.2-lt
s
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
9bc0e050dfa7 bridge bridge local
f36f52fc0987 docker_volumes-backup-extension-desktop-extension_default bridge local
67c79453bfb0 host host local
with following command run minikube cluster
minikube start
$ kubectl.exe cluster-info
Kubernetes control plane is running at https://127.0.0.1:55722
CoreDNS is running at https://127.0.0.1:55722/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl.exe get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION
CONTAINER-RUNTIME
minikube Ready control-plane 3h14m v1.28.3 192.168.49.2 <none> Ubuntu 22.04.3 LTS 5.15.153.1-microsoft-standard-W
SL2 docker://24.0.7
$ kubectl.exe get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready control-plane 3h15m v1.28.3
now run the following serviceaccount.yaml file on your kubernetes cluster(create new file with sa.yaml and copy following content)
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: jenkins
rules:
- apiGroups:
- '*'
resources:
- statefulsets
- services
- replicationcontrollers
- replicasets
- podtemplates
- podsecuritypolicies
- pods
- pods/log
- pods/exec
- podpreset
- poddisruptionbudget
- persistentvolumes
- persistentvolumeclaims
- jobs
- endpoints
- deployments
- deployments/scale
- daemonsets
- cronjobs
- configmaps
- namespaces
- events
- secrets
verbs:
- create
- get
- watch
- delete
- list
- patch
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts:jenkins
---
apiVersion: v1
kind: Secret
metadata:
name: jenkins-token
namespace: jenkins
annotations:
kubernetes.io/service-account.name: jenkins
type: kubernetes.io/service-account-token
first create jenkins namespace
kubectl create ns jenkins
kubectl.exe apply -f manifest/sa.yaml -n jenkins
kubectl.exe apply -f sa.yaml -n jenkins
now get the secrets
$ kubectl describe secret jenkins-token -n jenkins
Name: jenkins-token
Namespace: jenkins
Labels: <none>
Annotations: kubernetes.io/service-account.name: jenkins
kubernetes.io/service-account.uid: 178dbbe6-fbbc-4f5c-99d6-c7f05a170880
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1111 bytes
namespace: 7 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IjNIeVRGRUE5NHF3eVFaUEJHbUd0SkxRZExFQ2R1V2w0anpYSmI0Q0V5c0EifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJqZW5raW5zIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImplbmtpbnMtdG9rZW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiamVua2lucyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjE3OGRiYmU2LWZiYmMtNGY1Yy05OWQ2LWM3ZjA1YTE3MDg4MCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpqZW5raW5zOmplbmtpbnMifQ.RV8Jh3H8mp1_NMk7q_gVaqjsTC2GeI64G1yZKDUJ4DLQi76BvGB1JajkALIqgjMy43qqfESox9Wku0ZdF2nMcxniOPppPwwwcmGSBI7sy3PJdCAZ207QSm519e2-sq4h89YhEnvOvlf4aj7PU_qM-mhPH6SNsnCeXX6zX7Auv977otlqI8ne2sFMNon4osSFBdl78Q5YD22XjaKDCQ2vhZDYB9bBo6tYE9nLzHhxqu-liACuKfENPzv6KZ7JZXa2JiPqY8Cvl5qWlQEZATDu7XZsYPvpuGtfJnoeweLUv5GUlvUvyI7V4HAnRXFjTk2dcoHIJbpEtyjvngKVNAryog
Save this token somewhere
now lets goto jenkins , make sure you have following plugin installed
Or you can install it from Manage Jenkins --> Plugins ---> Available plugins ---> search and install
Once done go to Manage Jenkins --> Credentials ----> Add following credentials -----> Kind => secret text
with name kubernetes-jenkins-service-account-token
In the place of secrets add above token
then go to kubernetes setup
As our jenkins is running inside the docker container if it has to connect to kubernetes then it has to go through docker network
$ kubectl.exe cluster-info
Kubernetes control plane is running at https://127.0.0.1:55722
CoreDNS is running at https://127.0.0.1:55722/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
9bc0e050dfa7 bridge bridge local
f36f52fc0987 docker_volumes-backup-extension-desktop-extension_default bridge local
67c79453bfb0 host host local
cf2e652848df minikube bridge local
48b35c701b2b none null local
Test connection
Now to test a pipeline
- Create a pipeline job
Goto Jenkins Dashboard ---> New Item ----> Pipeline (give name like "sample-k8s-agent-pipeline") ----> save following groovy under "Pipeline script"
pipeline {
agent {
kubernetes {
yaml '''
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: some-label-value
spec:
containers:
- name: maven
image: maven:alpine
command:
- cat
tty: true
- name: busybox
image: busybox
command:
- cat
tty: true
'''
retries 2
}
}
stages {
stage('Run maven') {
steps {
container('maven') {
sh 'mvn -version'
}
container('busybox') {
sh '/bin/busybox'
}
}
}
}
}
Save and Build the pipeline
$ kubectl.exe get pod -n jenkins
NAME READY STATUS RESTARTS AGE
sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc 3/3 Running 0 39s
This is the console output
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] retry
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes jenkins/sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc
Still waiting to schedule task
‘sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc’ is offline
Agent sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc is provisioned from template sample-k8s-agent-pipeline_2-h9jth-j677f
---
apiVersion: "v1"
kind: "Pod"
metadata:
annotations:
kubernetes.jenkins.io/last-refresh: "1719134288606"
buildUrl: "http://host.minikube.internal:8080/job/sample-k8s-agent-pipeline/2/"
runUrl: "job/sample-k8s-agent-pipeline/2/"
labels:
some-label: "some-label-value"
jenkins: "slave"
jenkins/label-digest: "e9d0acef03990d8ff73dd8852424064ff4beefb0"
jenkins/label: "sample-k8s-agent-pipeline_2-h9jth"
kubernetes.jenkins.io/controller: "http___host_minikube_internal_8080x"
name: "sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc"
namespace: "jenkins"
spec:
containers:
- command:
- "cat"
image: "maven:alpine"
name: "maven"
tty: true
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- command:
- "cat"
image: "busybox"
name: "busybox"
tty: true
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- env:
- name: "JENKINS_SECRET"
value: "********"
- name: "JENKINS_AGENT_NAME"
value: "sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc"
- name: "REMOTING_OPTS"
value: "-noReconnectAfter 1d"
- name: "JENKINS_NAME"
value: "sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc"
- name: "JENKINS_AGENT_WORKDIR"
value: "/home/jenkins/agent"
- name: "JENKINS_URL"
value: "http://host.minikube.internal:8080/"
image: "jenkins/inbound-agent:3248.v65ecb_254c298-2"
name: "jnlp"
resources:
requests:
memory: "256Mi"
cpu: "100m"
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
nodeSelector:
kubernetes.io/os: "linux"
restartPolicy: "Never"
volumes:
- emptyDir:
medium: ""
name: "workspace-volume"
Running on sample-k8s-agent-pipeline-2-h9jth-j677f-xc8qc in /home/jenkins/agent/workspace/sample-k8s-agent-pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Run maven)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ mvn -version
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T19:00:29Z)
Maven home: /usr/share/maven
Java version: 1.8.0_212, vendor: IcedTea, runtime: /usr/lib/jvm/java-1.8-openjdk/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.153.1-microsoft-standard-wsl2", arch: "amd64", family: "unix"
[Pipeline] }
[Pipeline] // container
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ /bin/busybox
BusyBox v1.36.1 (2023-05-18 22:34:17 UTC) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2015.
Licensed under GPLv2. See source distribution for detailed
copyright notices.
Usage: busybox [function [arguments]...]
or: busybox --list[-full]
or: busybox --show SCRIPT
or: busybox --install [-s] [DIR]
or: function [arguments]...
BusyBox is a multi-call binary that combines many common Unix
utilities into a single executable. Most people will create a
link to busybox for each function they wish to use and BusyBox
will act like whatever it was invoked as.
Currently defined functions:
[, [[, acpid, add-shell, addgroup, adduser, adjtimex, ar, arch, arp,
arping, ascii, ash, awk, base32, base64, basename, bc, beep,
blkdiscard, blkid, blockdev, bootchartd, brctl, bunzip2, bzcat, bzip2,
cal, cat, chat, chattr, chgrp, chmod, chown, chpasswd, chpst, chroot,
chrt, chvt, cksum, clear, cmp, comm, conspy, cp, cpio, crc32, crond,
crontab, cryptpw, cttyhack, cut, date, dc, dd, deallocvt, delgroup,
deluser, depmod, devmem, df, dhcprelay, diff, dirname, dmesg, dnsd,
dnsdomainname, dos2unix, dpkg, dpkg-deb, du, dumpkmap, dumpleases,
echo, ed, egrep, eject, env, envdir, envuidgid, ether-wake, expand,
expr, factor, fakeidentd, fallocate, false, fatattr, fbset, fbsplash,
fdflush, fdformat, fdisk, fgconsole, fgrep, find, findfs, flock, fold,
free, freeramdisk, fsck, fsck.minix, fsfreeze, fstrim, fsync, ftpd,
ftpget, ftpput, fuser, getopt, getty, grep, groups, gunzip, gzip, halt,
hd, hdparm, head, hexdump, hexedit, hostid, hostname, httpd, hush,
hwclock, i2cdetect, i2cdump, i2cget, i2cset, i2ctransfer, id, ifconfig,
ifdown, ifenslave, ifplugd, ifup, inetd, init, insmod, install, ionice,
iostat, ip, ipaddr, ipcalc, ipcrm, ipcs, iplink, ipneigh, iproute,
iprule, iptunnel, kbd_mode, kill, killall, killall5, klogd, last, less,
link, linux32, linux64, linuxrc, ln, loadfont, loadkmap, logger, login,
logname, logread, losetup, lpd, lpq, lpr, ls, lsattr, lsmod, lsof,
lspci, lsscsi, lsusb, lzcat, lzma, lzop, makedevs, makemime, man,
md5sum, mdev, mesg, microcom, mim, mkdir, mkdosfs, mke2fs, mkfifo,
mkfs.ext2, mkfs.minix, mkfs.vfat, mknod, mkpasswd, mkswap, mktemp,
modinfo, modprobe, more, mount, mountpoint, mpstat, mt, mv, nameif,
nanddump, nandwrite, nbd-client, nc, netstat, nice, nl, nmeter, nohup,
nologin, nproc, nsenter, nslookup, ntpd, od, openvt, partprobe, passwd,
paste, patch, pgrep, pidof, ping, ping6, pipe_progress, pivot_root,
pkill, pmap, popmaildir, poweroff, powertop, printenv, printf, ps,
pscan, pstree, pwd, pwdx, raidautorun, rdate, rdev, readahead,
readlink, readprofile, realpath, reboot, reformime, remove-shell,
renice, reset, resize, resume, rev, rm, rmdir, rmmod, route, rpm,
rpm2cpio, rtcwake, run-init, run-parts, runlevel, runsv, runsvdir, rx,
script, scriptreplay, sed, seedrng, sendmail, seq, setarch, setconsole,
setfattr, setfont, setkeycodes, setlogcons, setpriv, setserial, setsid,
setuidgid, sh, sha1sum, sha256sum, sha3sum, sha512sum, showkey, shred,
shuf, slattach, sleep, smemcap, softlimit, sort, split, ssl_client,
start-stop-daemon, stat, strings, stty, su, sulogin, sum, sv, svc,
svlogd, svok, swapoff, swapon, switch_root, sync, sysctl, syslogd, tac,
tail, tar, taskset, tc, tcpsvd, tee, telnet, telnetd, test, tftp,
tftpd, time, timeout, top, touch, tr, traceroute, traceroute6, tree,
true, truncate, ts, tsort, tty, ttysize, tunctl, ubiattach, ubidetach,
ubimkvol, ubirename, ubirmvol, ubirsvol, ubiupdatevol, udhcpc, udhcpc6,
udhcpd, udpsvd, uevent, umount, uname, unexpand, uniq, unix2dos,
unlink, unlzma, unshare, unxz, unzip, uptime, users, usleep, uudecode,
uuencode, vconfig, vi, vlock, volname, w, wall, watch, watchdog, wc,
wget, which, who, whoami, whois, xargs, xxd, xz, xzcat, yes, zcat,
zcip
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // retry
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS
When you deploy the kubernetes cluster on AWS follow the following method
kubernetes setup:
Following is groovy script
pipeline {
agent {
kubernetes {
yaml '''
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: some-label-value
spec:
imagePullSecrets:
- name: regcred
containers:
- name: maven
image: nexus.theguru.in.net:8082/maven:1.0.0
command:
- cat
tty: true
- name: busybox
image: nexus.theguru.in.net:8082/busybox:1.0.0
command:
- cat
tty: true
'''
retries 2
}
}
stages {
stage('Run maven') {
steps {
container('maven') {
sh 'mvn -version'
}
container('busybox') {
sh '/bin/busybox'
}
}
}
}
}
I was getting following error with above setup it got resolved
Started by user unknown or anonymous
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] retry
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes jenkins/test1-8-j2qld-6pb8s-qx1tx
jenkins/test1-8-j2qld-6pb8s-qx1tx Container jnlp was terminated.
- busybox -- running
-----Logs-------------
- jnlp -- terminated (255)
-----Logs-------------
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.SslFilter.doHandshakeStep(SslFilter.java:540)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.SslFilter.processRead(SslFilter.java:360)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:111)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:113)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:295)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:279)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Unknown Source)
at java.base/sun.nio.ch.Invoker.invokeDirect(Unknown Source)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.implRead(Unknown Source)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(Unknown Source)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(Unknown Source)
at java.base/java.nio.channels.AsynchronousSocketChannel.read(Unknown Source)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter._read(TransportFilter.java:279)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$3.completed(TransportFilter.java:191)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$3.completed(TransportFilter.java:185)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Unknown Source)
at java.base/sun.nio.ch.Invoker$2.run(Unknown Source)
at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching jenkins.theguru.in.net found.
at java.base/sun.security.util.HostnameChecker.matchDNS(Unknown Source)
at java.base/sun.security.util.HostnameChecker.match(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
... 30 more
- maven -- running
-----Logs-------------
ERROR: Failed to launch test1-8-j2qld-6pb8s-qx1tx
java.lang.IllegalStateException: Containers are terminated with exit codes: {jnlp=255}
at PluginClassLoader for kubernetes//org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.checkTerminatedContainers(KubernetesLauncher.java:350)
at PluginClassLoader for kubernetes//org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:283)
at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:297)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
[Pipeline] // node
[Pipeline] }
Could not find a node block associated with node (source of error) but inside podTemplate
Queue task was cancelled
org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 953e2fbf-fd46-44fa-b42c-6bfde4cf5c88
Retrying
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes jenkins/test1-8-j2qld-6pb8s-kncx9
jenkins/test1-8-j2qld-6pb8s-kncx9 Container jnlp was terminated.
ERROR: Failed to launch test1-8-j2qld-6pb8s-kncx9
java.lang.IllegalStateException: Containers are terminated with exit codes: {jnlp=255}
at PluginClassLoader for kubernetes//org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.checkTerminatedContainers(KubernetesLauncher.java:350)
at PluginClassLoader for kubernetes//org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:283)
at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:297)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
- busybox -- running
-----Logs-------------
- jnlp -- terminated (255)
-----Logs-------------
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:113)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:295)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:279)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Unknown Source)
at java.base/sun.nio.ch.Invoker.invokeDirect(Unknown Source)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.implRead(Unknown Source)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(Unknown Source)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(Unknown Source)
at java.base/java.nio.channels.AsynchronousSocketChannel.read(Unknown Source)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter._read(TransportFilter.java:279)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$3.completed(TransportFilter.java:191)
at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$3.completed(TransportFilter.java:185)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Unknown Source)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(Unknown Source)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(Unknown Source)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(Unknown Source)
at java.base/sun.nio.ch.EPollPort$EventHandlerTask.run(Unknown Source)
at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching jenkins.theguru.in.net found.
at java.base/sun.security.util.HostnameChecker.matchDNS(Unknown Source)
at java.base/sun.security.util.HostnameChecker.match(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
... 33 more
- maven -- running
-----Logs-------------
[Pipeline] // node
[Pipeline] }
[Pipeline] // retry
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Queue task was cancelled
org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: b5d3af15-0b9a-4da5-b9d9-7d5b30faa8f5
Finished: ABORTED
After above setup it worked as follows
This was main culprit
Started by user unknown or anonymous
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] retry
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes jenkins/test1-10-pd6rq-qt5v5-791r1
Still waiting to schedule task
‘test1-10-pd6rq-qt5v5-791r1’ is offline
Agent test1-10-pd6rq-qt5v5-791r1 is provisioned from template test1_10-pd6rq-qt5v5
---
apiVersion: "v1"
kind: "Pod"
metadata:
annotations:
kubernetes.jenkins.io/last-refresh: "1731231498809"
buildUrl: "http://107.22.29.168:8080/job/test1/10/"
runUrl: "job/test1/10/"
labels:
some-label: "some-label-value"
jenkins: "slave"
jenkins/label-digest: "c6353536b084e100d71a439f16eaa9b97cb82790"
jenkins/label: "test1_10-pd6rq"
kubernetes.jenkins.io/controller: "http___107_22_29_168_8080x"
name: "test1-10-pd6rq-qt5v5-791r1"
namespace: "jenkins"
spec:
containers:
- command:
- "cat"
image: "maven:alpine"
name: "maven"
tty: true
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- command:
- "cat"
image: "busybox"
name: "busybox"
tty: true
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- env:
- name: "JENKINS_SECRET"
value: "********"
- name: "JENKINS_AGENT_NAME"
value: "test1-10-pd6rq-qt5v5-791r1"
- name: "JENKINS_WEB_SOCKET"
value: "true"
- name: "REMOTING_OPTS"
value: "-noReconnectAfter 1d"
- name: "JENKINS_NAME"
value: "test1-10-pd6rq-qt5v5-791r1"
- name: "JENKINS_AGENT_WORKDIR"
value: "/home/jenkins/agent"
- name: "JENKINS_URL"
value: "http://107.22.29.168:8080/"
image: "jenkins/inbound-agent:3261.v9c670a_4748a_9-2"
name: "jnlp"
resources:
requests:
memory: "256Mi"
cpu: "100m"
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
imagePullSecrets:
- name: "regcred"
nodeSelector:
kubernetes.io/os: "linux"
restartPolicy: "Never"
volumes:
- emptyDir:
medium: ""
name: "workspace-volume"
Running on test1-10-pd6rq-qt5v5-791r1 in /home/jenkins/agent/workspace/test1
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Run maven)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ mvn -version
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T19:00:29Z)
Maven home: /usr/share/maven
Java version: 1.8.0_212, vendor: IcedTea, runtime: /usr/lib/jvm/java-1.8-openjdk/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.10.227-219.884.amzn2.x86_64", arch: "amd64", family: "unix"
[Pipeline] }
[Pipeline] // container
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ /bin/busybox
BusyBox v1.37.0 (2024-09-26 21:31:42 UTC) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2015.
Licensed under GPLv2. See source distribution for detailed
copyright notices.
Usage: busybox [function [arguments]...]
or: busybox --list[-full]
or: busybox --show SCRIPT
or: busybox --install [-s] [DIR]
or: function [arguments]...
BusyBox is a multi-call binary that combines many common Unix
utilities into a single executable. Most people will create a
link to busybox for each function they wish to use and BusyBox
will act like whatever it was invoked as.
Currently defined functions:
[, [[, acpid, add-shell, addgroup, adduser, adjtimex, ar, arch, arp,
arping, ascii, ash, awk, base32, base64, basename, bc, beep,
blkdiscard, blkid, blockdev, bootchartd, brctl, bunzip2, bzcat, bzip2,
cal, cat, chat, chattr, chgrp, chmod, chown, chpasswd, chpst, chroot,
chrt, chvt, cksum, clear, cmp, comm, conspy, cp, cpio, crc32, crond,
crontab, cryptpw, cttyhack, cut, date, dc, dd, deallocvt, delgroup,
deluser, depmod, devmem, df, dhcprelay, diff, dirname, dmesg, dnsd,
dnsdomainname, dos2unix, dpkg, dpkg-deb, du, dumpkmap, dumpleases,
echo, ed, egrep, eject, env, envdir, envuidgid, ether-wake, expand,
expr, factor, fakeidentd, fallocate, false, fatattr, fbset, fbsplash,
fdflush, fdformat, fdisk, fgconsole, fgrep, find, findfs, flock, fold,
free, freeramdisk, fsck, fsck.minix, fsfreeze, fstrim, fsync, ftpd,
ftpget, ftpput, fuser, getfattr, getopt, getty, grep, groups, gunzip,
gzip, halt, hd, hdparm, head, hexdump, hexedit, hostid, hostname,
httpd, hush, hwclock, i2cdetect, i2cdump, i2cget, i2cset, i2ctransfer,
id, ifconfig, ifdown, ifenslave, ifplugd, ifup, inetd, init, insmod,
install, ionice, iostat, ip, ipaddr, ipcalc, ipcrm, ipcs, iplink,
ipneigh, iproute, iprule, iptunnel, kbd_mode, kill, killall, killall5,
klogd, last, less, link, linux32, linux64, linuxrc, ln, loadfont,
loadkmap, logger, login, logname, logread, losetup, lpd, lpq, lpr, ls,
lsattr, lsmod, lsof, lspci, lsscsi, lsusb, lzcat, lzma, lzop, makedevs,
makemime, man, md5sum, mdev, mesg, microcom, mim, mkdir, mkdosfs,
mke2fs, mkfifo, mkfs.ext2, mkfs.minix, mkfs.vfat, mknod, mkpasswd,
mkswap, mktemp, modinfo, modprobe, more, mount, mountpoint, mpstat, mt,
mv, nameif, nanddump, nandwrite, nbd-client, nc, netstat, nice, nl,
nmeter, nohup, nologin, nproc, nsenter, nslookup, ntpd, od, openvt,
partprobe, passwd, paste, patch, pgrep, pidof, ping, ping6,
pipe_progress, pivot_root, pkill, pmap, popmaildir, poweroff, powertop,
printenv, printf, ps, pscan, pstree, pwd, pwdx, raidautorun, rdate,
rdev, readahead, readlink, readprofile, realpath, reboot, reformime,
remove-shell, renice, reset, resize, resume, rev, rm, rmdir, rmmod,
route, rpm, rpm2cpio, rtcwake, run-init, run-parts, runlevel, runsv,
runsvdir, rx, script, scriptreplay, sed, seedrng, sendmail, seq,
setarch, setconsole, setfattr, setfont, setkeycodes, setlogcons,
setpriv, setserial, setsid, setuidgid, sh, sha1sum, sha256sum, sha3sum,
sha512sum, showkey, shred, shuf, slattach, sleep, smemcap, softlimit,
sort, split, ssl_client, start-stop-daemon, stat, strings, stty, su,
sulogin, sum, sv, svc, svlogd, svok, swapoff, swapon, switch_root,
sync, sysctl, syslogd, tac, tail, tar, taskset, tc, tcpsvd, tee,
telnet, telnetd, test, tftp, tftpd, time, timeout, top, touch, tr,
traceroute, traceroute6, tree, true, truncate, ts, tsort, tty, ttysize,
tunctl, ubiattach, ubidetach, ubimkvol, ubirename, ubirmvol, ubirsvol,
ubiupdatevol, udhcpc, udhcpc6, udhcpd, udpsvd, uevent, umount, uname,
unexpand, uniq, unix2dos, unlink, unlzma, unshare, unxz, unzip, uptime,
users, usleep, uudecode, uuencode, vconfig, vi, vlock, volname, w,
wall, watch, watchdog, wc, wget, which, who, whoami, whois, xargs, xxd,
xz, xzcat, yes, zcat, zcip
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // retry
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS
You mey get following error if you install the kubernetes plugin and don,t restart the the kubernetes
First time build. Skipping changelog.
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 4: Invalid agent type "kubernetes" specified. Must be one of [any, label, none] @ line 4, column 9.
kubernetes {
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1107)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
at PluginClassLoader for script-security//org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:190)
at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:175)
at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:652)
at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:598)
at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:335)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:445)
Finished: FAILURE
If you get following error
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 72s default-scheduler 0/2 nodes are available: 2 node(s) didn't match Pod's node affinity/selector. pre
emption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling..
Do this
Dell@DESKTOP-CL9FNGA MINGW64 ~
$ kubectl.exe get nodes
NAME STATUS ROLES AGE VERSION
ip-10-0-101-209.ec2.internal Ready <none> 14h v1.28.13-eks-a737599
ip-10-0-2-91.ec2.internal Ready <none> 14h v1.28.13-eks-a737599
Dell@DESKTOP-CL9FNGA MINGW64 ~
$ kubectl label nodes ip-10-0-101-209.ec2.internal buildRole=true
node/ip-10-0-101-209.ec2.internal labeled
Dell@DESKTOP-CL9FNGA MINGW64 ~
$ kubectl label nodes ip-10-0-2-91.ec2.internal buildRole=true
node/ip-10-0-2-91.ec2.internal labeled
0
Subscribe to my newsletter
Read articles from Anant Saraf directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by