Join Dedicated Ingress Node to RKE2 Cluster

In this article I will show you how to Join new node as Dedicated Ingress Node in existing RKE2 cluster. Ingress node is custom role to handle external/ingress traffic. In this node, no pods and applications running except ingress controller.

So, let's get started…

Environment

• Master Node

• Worker Node

• Ingress Node

Join Ingress Node to RKE2 Cluster

1. Mapping hosts

# exec on all nodes
nano /etc/hosts
---
172.20.20.65 at-rke2-1 at-rke2-1.at.lab rancher.at.lab
172.20.20.66 at-rke2-2 at-rke2-2.at.lab
172.20.20.67 at-rke2-3 at-rke2-3.at.lab

2. Update and upgrade packages

# exec on at-rke-3 nodes
apt-get update -y && apt-get upgrade -y

3. Disable swap

# exec on at-rke-3 nodes
systemctl disable --now swap.target
swapoff -a

4. Enable containerd modules

# exec on at-rke-3 nodes
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter

5. Configure kernel settings

# exec on at-rke-3 nodes
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system

6. Create rke2 agent configuration file

# exec on at-rke-3 nodes
mkdir -p /etc/rancher/rke2
nano /etc/rancher/rke2/config.yaml
---
server: https://at-rke2-1.at.lab:9345
write-kubeconfig-mode: "0644"
token: [REDACTED - SAME WITH MASTER NODE]
tls-san:
  - rancher.at.lab
  - at-rke2-1.at.lab
  - at-rke2-2.at.lab
  - at-rke2-3.at.lab

7. Update rke2 configuration file with add at-rke2-3 tls in other nodes

# exec on at-rke-1 & at-rke-2 node
nano /etc/rancher/rke2/config.yaml
---
tls-san:
  - rancher.at.lab
  - at-rke2-1.at.lab
  - at-rke2-2.at.lab
  - at-rke2-3.at.lab
...

8. Bootstrap rke2 cluster

# exec on at-rke-3 node
curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" INSTALL_RKE2_VERSION=v1.32.7+rke2r1 sh -

9. Enable and start rke2-agent

# exec on at-rke-3 node
systemctl enable rke2-agent.service
systemctl start rke2-agent.service

10. Set ingress role to at-rke2-3 node

# exec on at-rke2-1 node
kubectl label node at-rke2-3 node-role.kubernetes.io/ingress="true"

11. Set NoSchedule taint to at-rke2-3 or ingress node

# exec on at-rke-1 node
kubectl taint node at-rke2-3 node-role.kubernetes.io/ingress="true":NoSchedule

12. Configure ingress-nginx-controller to run only on ingress node

# exec on at-rke-1 node
kubectl -n kube-system edit daemonset rke2-ingress-nginx-controller 
...
spec:
  template:
    spec:
      nodeSelector:
        node-role.kubernetes.io/ingress: "true"
      tolerations:
      - effect: NoSchedule
        key: "node-role.kubernetes.io/ingress"
        operator: Equal
        value: "true"

13. Verification

# exec on at-rke-1 node
kubectl get nodes -o wide
kubectl get daemonset --all-namespaces
kubectl get pods --all-namespaces -o wide | grep ingress-nginx-controller

14. Operational Test

# exec on at-rke-1 node
kubectl create deployment nginx-ingress --image=nginx --replicas=1
kubectl expose deployment nginx-ingress --port=80 --target-port=80 
kubectl create ingress nginx-ingress --class=nginx --rule="nginx-ingress.at.lab/*=nginx-ingress:80"

nano /etc/hosts
---
# note: mapping rancher just want to access via external/ingress, all nodes still mapping to server/master node
172.20.20.67 nginx-ingress.at.lab rancher.at.lab

Thank You.