How to Build a Homelab Using K3s

Setting up a Kubernetes Cluster might seem intimidating, especially if you're new to infrastructure and using macOS. But don't worry!

I will walk you through every step of how I created a Kubernetes Cluster using K3s and UTM Virtual Machines, all on my Mac

Whether you are a beginner to DevOps or someone who is trying to simulate a real infrastructure for testing, this setup will be the most feasible for you. Before we begin, I'd like to provide an overview of UTM and K3s, as well as the reasons why we need them.

Why do we need to use UTM?

UTM is free and open source, which means it can be easily used for building a home lab
It is specifically designed for macOS with a straightforward GUI for beginners to use. Yes!. Just install and Boom!!!
No Special permission is needed to use UTM because it runs on “Apple's Hypervisor virtualization framework,” which, in simple terms, means UTM runs in user space. Therefore, it does not need to add/write kernel extensions.
Under the hood, UTM uses QEMU, a very old and powerful virtualization engine that supports different CPU architectures like ARM and x86_64.

Now you all have heard of k8s, but what is k3s?

“K3s” is a lightweight and minimal version of Kubernetes (K8s), which is designed to use very little CPU and memory. It simplifies or removes complicated components like etcd (for storage) and uses the default lightweight SQLite for storing cluster info. As it comes with a single binary, it makes k3s very easy to install and upgrade.

So, for testing and creating a homelab, K3s will be our partner.

Assuming you have installed UTM on a Mac, the next step is to download an OS image, which we will use to create VMs. In my case, I have used the Ubuntu image (Ubuntu 24.04.2 LTS). You can download the iso from here.

After this, the steps are:

STEP 1:

When you open the UTM app, you will see something like this:

Note: Ignore the Nodes created. Initially, the node list will be empty, and you have to create.

Click on “Create a New Virtual Machine” and then on “Virtualize”.
Then click on “Linux”
Then get the ISO (downloaded before) and click on next
Set the CPU cores (2 is enough for Ubuntu)
Set the memory to around 4 GB
Set the size of the drive to around 64 GB
Click on save

STEP 2:

Run the newly created instance, which will install the Linux
A very important step after installing is to close the VM and remove the “Ubuntu ISO” from the CD/DVD

Click on “clear” and it will remove the linkage

STEP 3 (Installing k3s on master node):

For simplicity, download two VMs and name one machine “master-node” and the other “worker-node-1”.
Run the “master-node” VM and open the terminal.
Firstly, to update each package installed, type “sudo apt update”. This updates each package, ensuring every package/module is the latest.
Enter “sudo ufw disable”. This turns off the uncomplicated firewall
Use “curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--flannel-backend=none --disable-network-policy" sh -”.

Note: I have decided not to use “flannel” as it is the default CNI, and this will not provide us with the flexibility to learn networking in depth. We will use “Calico” as the CNI to attach the worker nodes later

To link/attach other VMs as a worker node, we need the K3s Token.
type “sudo cat /var/lib/rancher/k3s/server/node-token”. This is the path where the token is stored, and the “cat” command is used to list the content
now type “ip a | grep inet”. It will give us the IP address of the master node.
Log in/ssh to the worker node created.
Here, also turn off the firewall by using “sudo ufw disable”
Open the terminal and paste this: curl -sfL https://get.k3s.io | K3S_URL=https://<MASTER_IP>:6443 K3S_TOKEN=<TOKEN> sh -”. (The K3s server/ master node needs 6443 to be accessible by other worker nodes/agents.)
After the K3s gets installed in the worker node, go back to the master node and type “sudo kubectl get nodes”
You will see “worker-node” but the status will be “Not Ready” because there is no CNI installed

STEP 4: (Installing Calico)

Go to Master Node and paste this in the terminal: “ kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.2/manifests/calico.yaml”
If you type “sudo kubectl get pods” you will see this:
Now, if you do “sudo kubectl get nodes -o wide”, you see this:
Now paste this in the terminal: “sudo vim /etc/cni/net.d/10-calico.conflist”.
add the ip-forewarding setting: "container_settings": { "allow_ip_forwarding": true }
Save and close the file using “wq!”.
If you now type “ sudo cat /etc/cni/net.d/10-calico.conflist”, you see the ip-forewarding is now set to true
Congrats!!!. Calico is installed and the nodes are connected :)

In conclusion, setting up a lightweight Kubernetes cluster using K3s and UTM virtual machines on macOS is an accessible and efficient way to simulate real infrastructure for testing and learning purposes. By leveraging UTM's user-friendly interface and K3s's minimal resource requirements, even beginners can create a functional Kubernetes environment. This setup not only provides a practical learning experience but also offers flexibility in exploring different networking configurations, such as using Calico as the CNI. With these tools, you can effectively build and manage a home lab, enhancing your understanding of Kubernetes and its components.

Building a Lightweight Kubernetes Cluster with K3s and UTM VMs (MacOS)