Upgrading kubeadm cluster - CKA Exam
mehdi pasha8 min read
During my CKA exam one of the question was to upgrade the cluster only master node from 1.30.0 to 1.30.1
Below is the official doc link I used during my exam.
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/
Note:- While attempting the exam, it was initially little confusing make sure you check which node the question mention to upgrade (ssh to node and use sudo -i ) as mentioned.
Steps to perform upgrade .
Step 1 : First check nodes
controlplane ~ ➜ k get nodes
NAME STATUS ROLES AGE VERSION
controlplane Ready control-plane 33m v1.30.0
node01 Ready <none> 32m v1.30.0
Step 2 : Changing the package repository
This step is to verify that the package is repository is correct , in my case it is latest repostory but if the question comes with other version to upgrade make sure once check this step.
# On your system, this configuration file could have a different name
pager /etc/apt/sources.list.d/kubernetes.list
deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /
Step 3 : Version Upgrade kubeadm
Determine which version to upgrade .
Find the latest patch release for Kubernetes 1.30 using the OS package manager:
# Find the latest 1.30 version in the list.
# It should look like 1.30.x-*, where x is the latest patch.
sudo apt update
sudo apt-cache madison kubeadm
This command will give you detail above version you can upgrade .
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease [1,186 B]
Get:3 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb Packages [6,856 B]
Get:4 http://security.ubuntu.com/ubuntu bionic-security InRelease [102 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [102 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [102 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,411 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,786 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1,728 kB]
Get:14 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1,688 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [30.8 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [64.0 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [20.6 kB]
Get:18 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [23.8 kB]
Get:19 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,637 kB]
Get:20 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [3,373 kB]
Fetched 28.2 MB in 5s (5,191 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
student-node ~ ➜ sudo apt-cache madison kubeadm
kubeadm | 1.30.3-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
kubeadm | 1.30.2-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
kubeadm | 1.30.1-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
kubeadm | 1.30.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
Upgrading master node
For the control plane node
1 .Upgrade kubeadm:
Make sure you understand the version below is example and select the version you need to upgrade you get this info from sudo apt-cache madison kubeadm ( Here i am focusing because sometimes it create confusion which version to upgrade how to understand kubeadm version)
kubeadm | 1.30.3-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
kubeadm | 1.30.2-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
kubeadm | 1.30.1-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
kubeadm | 1.30.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages
controlplane ~ ➜ sudo apt-mark unhold kubeadm && \
sudo apt-get update && sudo apt-get install -y kubeadm='1.30.1-1.1' && \
sudo apt-mark hold kubeadm
Canceled hold on kubeadm.
Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:3 https://download.docker.com/linux/ubuntu jammy InRelease
Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease
Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease
Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be upgraded:
kubeadm
1 upgraded, 0 newly installed, 0 to remove and 19 not upgraded.
Need to get 10.4 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubeadm 1.30.1-1.1 [10.4 MB]
Fetched 10.4 MB in 0s (36.8 MB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 20584 files and directories currently installed.)
Preparing to unpack .../kubeadm_1.30.1-1.1_amd64.deb ...
Unpacking kubeadm (1.30.1-1.1) over (1.30.0-1.1) ...
Setting up kubeadm (1.30.1-1.1) ...
kubeadm set on hold.
2 Verify that the download works and has the expected version:
controlplane ~ ➜ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"30", GitVersion:"v1.30.1", GitCommit:"6911225c3f747e1cd9d109c305436d08b668f086", GitTreeState:"clean", BuildDate:"2024-05-14T10:49:05Z", GoVersion:"go1.22.2", Compiler:"gc", Platform:"linux/amd64"}
3 Verify the upgrade plan: ( in exam you might try to save time and if sure skip this step)
controlplane ~ ➜ sudo kubeadm upgrade plan
[preflight] Running pre-flight checks.
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[upgrade] Running cluster health checks
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: 1.30.0
[upgrade/versions] kubeadm version: v1.30.1
[upgrade/versions] Target version: v1.30.3
[upgrade/versions] Latest version in the v1.30 series: v1.30.3
4.Choose a version to upgrade to, and run the appropriate command.
controlplane ~ ➜ sudo kubeadm upgrade apply v1.30.1
[preflight] Running pre-flight checks.
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[upgrade] Running cluster health checks
[upgrade/version] You have chosen to change the cluster version to "v1.30.1"
[upgrade/versions] Cluster version: v1.30.0
[upgrade/versions] kubeadm version: v1.30.1
[upgrade] Are you sure you want to proceed? [y/N]: y
[upgrade/prepull] Pulling images required for setting up a Kubernetes cluster
[upgrade/prepull] This might take a minute or two, depending on the speed of your internet connection
[upgrade/prepull] You can also perform this action in beforehand using 'kubeadm config images pull'
W0808 10:36:03.327403 27970 checks.go:844] detected that the sandbox image "registry.k8s.io/pause:3.8" of the container runtime is inconsistent with that used by kubeadm.It is recommended to use "registry.k8s.io/pause:3.9" as the CRI sandbox image.
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.30.1" (timeout: 5m0s)...
[upgrade/etcd] Upgrading to TLS for etcd
[upgrade/staticpods] Preparing for "etcd" upgrade
[upgrade/staticpods] Current and new manifests of etcd are equal, skipping upgrade
[upgrade/etcd] Waiting for etcd to become available
[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests646568571"
[upgrade/staticpods] Preparing for "kube-apiserver" upgrade
[upgrade/staticpods] Renewing apiserver certificate
[upgrade/staticpods] Renewing apiserver-kubelet-client certificate
[upgrade/staticpods] Renewing front-proxy-client certificate
[upgrade/staticpods] Renewing apiserver-etcd-client certificate
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-08-10-36-12/kube-apiserver.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/staticpods] This can take up to 5m0s
[apiclient] Found 1 Pods for label selector component=kube-apiserver
[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!
[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade
[upgrade/staticpods] Renewing controller-manager.conf certificate
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-08-10-36-12/kube-controller-manager.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/staticpods] This can take up to 5m0s
[apiclient] Found 1 Pods for label selector component=kube-controller-manager
[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!
[upgrade/staticpods] Preparing for "kube-scheduler" upgrade
[upgrade/staticpods] Renewing scheduler.conf certificate
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-08-08-10-36-12/kube-scheduler.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[upgrade/staticpods] This can take up to 5m0s
[apiclient] Found 1 Pods for label selector component=kube-scheduler
[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config2378742340/config.yaml
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-prox
5.Once the command finishes you should see
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.30.1". Enjoy!
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
6.Drain the node ( Must execute step)
controlplane ~ ✖ kubectl drain controlplane --ignore-daemonsets
node/controlplane already cordoned
Warning: ignoring DaemonSet-managed Pods: kube-flannel/kube-flannel-ds-lrj7c, kube-system/kube-proxy-vfjj4
node/controlplane drained
7.Upgrade kubelet and kubectl
controlplane ~ ✖ sudo apt-mark unhold kubelet kubectl && \
sudo apt-get update && sudo apt-get install -y kubelet='1.30.1-1.1*' kubectl='1.30.1-1.1' && \
sudo apt-mark hold kubelet kubectl
Canceled hold on kubelet.
Canceled hold on kubectl.
Hit:2 https://download.docker.com/linux/ubuntu jammy InRelease
Hit:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb InRelease
Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease
Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Selected version '1.30.1-1.1' (isv:kubernetes:core:stable:v1.30:pkgs.k8s.io [amd64]) for 'kubelet'
The following packages will be upgraded:
kubectl kubelet
2 upgraded, 0 newly installed, 0 to remove and 18 not upgraded.
Need to get 28.9 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubectl 1.30.1-1.1 [10.8 MB]
Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubelet 1.30.1-1.1 [18.1 MB]
Fetched 28.9 MB in 1s (25.1 MB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 20584 files and directories currently installed.)
Preparing to unpack .../kubectl_1.30.1-1.1_amd64.deb ...
Unpacking kubectl (1.30.1-1.1) over (1.30.0-1.1) ...
Preparing to unpack .../kubelet_1.30.1-1.1_amd64.deb ...
Unpacking kubelet (1.30.1-1.1) over (1.30.0-1.1) ...
Setting up kubectl (1.30.1-1.1) ...
Setting up kubelet (1.30.1-1.1) ...
kubelet set on hold.
kubectl set on hold.
8.Restart the kubelet:
controlplane ~ ➜ sudo systemctl daemon-reload
sudo systemctl restart kubelet
9.Uncordon the node
controlplane ~ ➜ k get nodes
NAME STATUS ROLES AGE VERSION
controlplane Ready,SchedulingDisabled control-plane 46m v1.30.1
node01 Ready <none> 45m v1.30.0
controlplane ~ ➜ kubectl uncordon controlplane
node/controlplane uncordoned
10.Verify the upgrade
controlplane ~ ➜ k get nodes
NAME STATUS ROLES AGE VERSION
controlplane Ready control-plane 47m v1.30.1
node01 Ready <none> 47m v1.30.0
I didn't upgrade worker node here as it was not asked in the exam you can follow official doc to upgrade worker node
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/#upgrade-worker-nodes
0
Subscribe to my newsletter
Read articles from mehdi pasha directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by