Step-by-Step Guide to Setting Up a Kubernetes Cluster with Kubeadm

Setting up a Kubernetes (K8s) cluster with kubeadm involves important steps to configure both the control plane and worker nodes effectively. In this guide, we'll go through the entire process, highlighting key components and how they work together to create a fully functional K8s cluster.

Here’s an overview of these steps and how they work:

Complete Steps in K8s kubeadm Cluster Setup:

Prepare the Environment for Master and Worker Nodes: Ensure the OS is supported (e.g., Ubuntu 20.04) and meets hardware requirements (e.g., RAM, CPU). Disable swap.
Install Required Packages:
- Container Runtime: Install a container runtime (e.g., containerd) on both master and worker nodes.
- Kubernetes Components: Install kubelet, kubeadm, and kubectl.
Configure System Settings:
- Kernel Modules and Sysctl Settings: Load necessary kernel modules and set sysctl parameters to enable proper networking.
Initialize the Master Node:
- kubeadm init: Run this command on the master node to set up the control plane components. This will start the API server, scheduler, and controller manager.
Install the Pod Network (CNI Plugin):
- Weave Net Installation: After initializing the master node, install a CNI plugin like Weave Net to handle networking between pods. This is critical for inter-pod communication.
Set Up kubectl Access:
- Configuration for Users: Copy the Kubernetes admin configuration file to the home directories of both the root user and the non-root user (if run with sudo), allowing them to use kubectl.
Display Join Command for Worker Nodes:
- Join Command: Use kubeadm token create --print-join-command to get the command needed for worker nodes to join the cluster. This command includes a token and hash for secure communication.
Join Worker Nodes to the Cluster

kubeadm join: On each worker node, run the join command provided by the master node. This connects the worker node to the master and allows it to start running pods.
Verify the Cluster

Check Node Status: Use kubectl get nodes to ensure all nodes (master and workers) are registered and in a Ready state.

Deploy a Test Application: Deploy a simple application to verify functionality.

Minimum Configuration Requirements:

To set up the Kubernetes cluster, I recommend the following minimum configuration:

1. Master Nodes:

- Instance Type: t2.medium (2 vCPUs, 4GB RAM)

- Number of Nodes: 1

- Operating System: Ubuntu 20.04 LTS

- Storage: 20GB EBS (gp2)

2. Worker Nodes:

- Instance Type: t2.small(1 vCPUs, 2GB RAM)

- Number of Nodes: 2

- Operating System: Ubuntu 20.04 LTS

- Storage: 10GB EBS (gp2)

Setting Up the Cluster on AWS:

1. Launch EC2 Instances:

a. Launch one instance for master nodes using the recommended configuration.

b. Launch two instances for worker nodes using the recommended configuration.

2. Configure Security Groups:

A shell script for K8s Master Node set up:

#!/bin/bash
# ------------------------------------------------------------------------------
# Kubernetes Master Node Setup Script
# ------------------------------------------------------------------------------
# This script installs Kubernetes (kubeadm, kubelet, kubectl) on an Ubuntu server
# with containerd as the container runtime and configures Weave Net as the CNI plugin.
# It includes system validation, CRI configuration, and networking setup.
# Author: S.SUBBA REDDY
# Date: 06-09-2024
# ------------------------------------------------------------------------------

# Exit immediately if a command exits with a non-zero status
set -e

# Function to print messages
log_message() {
    echo -e "\e[32m[INFO] $1\e[0m"
}

# Function to print error messages
log_error() {
    echo -e "\e[31m[ERROR] $1\e[0m" >&2
    exit 1
}

# Function to check if the script is being run as root
check_root() {
    if [ "$(id -u)" -ne 0 ]; then
        log_error "This script must be run as root. Please use sudo."
    fi
    log_message "Running as root user."
}

# Function to disable swap
disable_swap() {
    if [ "$(swapon --show)" ]; then
        log_message "Disabling swap..."
        swapoff -a
        sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
    else
        log_message "Swap is already disabled."
    fi
}

# Function to load required kernel modules
load_kernel_modules() {
    log_message "Loading necessary kernel modules..."
    cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
    modprobe overlay
    modprobe br_netfilter
}

# Function to configure sysctl parameters
configure_sysctl() {
    log_message "Applying sysctl settings for networking..."
    cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
    sysctl --system
}

# Function to install containerd
install_containerd() {
    log_message "Installing containerd as the container runtime..."
    apt-get update -y
    apt-get install -y ca-certificates curl gnupg lsb-release

    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
    echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

    apt-get update -y
    apt-get install -y containerd.io

    # Configure containerd to use systemd as the cgroup driver
    containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
    sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

    # Restart containerd
    systemctl restart containerd
    systemctl enable containerd
}

# Function to install Kubernetes components
install_kubernetes() {
    log_message "Installing Kubernetes components (kubelet, kubeadm, kubectl)..."
    apt-get update -y
    apt-get install -y apt-transport-https ca-certificates curl

    curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
    echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

    apt-get update -y
    apt-get install -y kubelet kubeadm kubectl
    apt-mark hold kubelet kubeadm kubectl

    systemctl enable kubelet.service
}

# Function to initialize the Kubernetes master node
initialize_master_node() {
    log_message "Initializing Kubernetes master node..."
    # 10. Initialize Kubernetes master node (without Pod Network CIDR for Weave Net)
    kubeadm init --cri-socket /run/containerd/containerd.sock

    # Set up kubectl for the root user
    log_message "Configuring kubectl for root user..."
    mkdir -p /root/.kube
    cp -i /etc/kubernetes/admin.conf /root/.kube/config

    # If script was run with sudo, configure for the non-root user
    if [ "$SUDO_USER" ]; then
        user_home=$(eval echo ~$SUDO_USER)
        log_message "Configuring kubectl for non-root user: $SUDO_USER"
        mkdir -p "$user_home/.kube"
        cp -i /etc/kubernetes/admin.conf "$user_home/.kube/config"
        chown $(id -u $SUDO_USER):$(id -g $SUDO_USER) "$user_home/.kube/config"
    else
        log_message "No non-root user detected. Only root user is configured."
    fi
}


# Function to install the Weave Net CNI plugin
install_weave_net() {
    log_message "Installing Weave Net as the CNI plugin..."
    kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml
}

# Function to display the join command for worker nodes
display_join_command() {
    log_message "To add worker nodes, use the following join command:"
    kubeadm token create --print-join-command
}

# Main script execution
log_message "Starting Kubernetes master node setup..."

check_root
disable_swap
load_kernel_modules
configure_sysctl
install_containerd
install_kubernetes
initialize_master_node
install_weave_net
display_join_command

log_message "Kubernetes master node setup complete."

A shell script for K8s Worker Node set up:

#!/bin/bash

# ---------------------------------------------------------------------------
# Kubernetes Worker Node Setup Script with kubectl access for root and non-root
# ---------------------------------------------------------------------------
# This script sets up a Kubernetes worker node on an Ubuntu system and joins
# it to the Kubernetes master node using the join token.
# It ensures both root and the current non-root user (if the script is run
# with sudo) can access kubectl.
# Author: S.SUBBA REDDY
# Date: 06-09-2024
# Minimum EC2 Requirements for Worker Nodes Setup:
# Instance: t2.small (1 vCPU, 2 GB RAM)
# Storage: 8-10 GB SSD
# OS: Ubuntu 20.04
# ---------------------------------------------------------------------------


# Function to log messages
log_message() {
    echo "[INFO] $1"
}

# Function to log errors
log_error() {
    echo "[ERROR] $1" >&2
    exit 1
}

# Function to check if the script is being run as root
check_root() {
    if [ "$(id -u)" -ne 0 ]; then
        log_error "This script must be run as root. Please use sudo."
    fi
    log_message "Running as root user."
}

# Function to disable swap
disable_swap() {
    if [ "$(swapon --show)" ]; then
        log_message "Disabling swap..."
        swapoff -a
        sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
    else
        log_message "Swap is already disabled."
    fi
}

# Function to load kernel modules and configure sysctl settings
configure_sysctl() {
    log_message "Configuring kernel modules and sysctl settings..."

    cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

    modprobe overlay
    modprobe br_netfilter

    cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF

    sysctl --system
}

# Function to install containerd
install_containerd() {
    log_message "Installing containerd..."

    apt-get update -y
    apt-get install -y ca-certificates curl gnupg lsb-release

    mkdir -p /etc/apt/keyrings
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
    echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
    apt-get update -y
    apt-get install -y containerd.io

    log_message "Configuring containerd to use systemd as the cgroup driver..."
    containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
    sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

    systemctl restart containerd
    systemctl enable containerd
}

# Function to install Kubernetes components
install_kubernetes() {
    log_message "Installing Kubernetes components..."

    apt-get update
    apt-get install -y apt-transport-https ca-certificates curl
    curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
    echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

    apt-get update
    apt-get install -y kubelet kubeadm kubectl
    apt-mark hold kubelet kubeadm kubectl
}

# Function to enable and start kubelet service
start_kubelet() {
    log_message "Enabling and starting kubelet service..."

    systemctl daemon-reload
    systemctl start kubelet
    systemctl enable kubelet.service
}

# Function to join the Kubernetes cluster
join_cluster() {
    local join_command="$1"
    if [ -z "$join_command" ]; then
        log_error "Join command not provided. Please provide the join command from the master node."
    fi
    log_message "Joining the Kubernetes cluster..."
    kubeadm join $join_command
}

### Main script starts here ###

check_root
disable_swap
configure_sysctl
install_containerd
install_kubernetes
start_kubelet

# Replace <JOIN_COMMAND> with the actual join command from the master node
JOIN_COMMAND=" "
join_cluster "$JOIN_COMMAND"

log_message "Kubernetes worker node setup complete."

By following these steps, you can set up a fully operational Kubernetes cluster using kubeadm, ensuring that networking, resource scaling, and monitoring are properly configured for managing containerized applications.

Master Node:

Worker-Node1:

Worker-Node2:

How It Works

Master Node Control Plane: The master node manages the entire cluster, ensuring that the desired state of the applications is maintained.
Worker Nodes: Worker nodes run the actual application workloads, reporting back to the master about their status.
Networking: The Weave Net plugin handles network communication between pods across nodes.
Load Balancing and Scaling: Kubernetes balances workloads automatically and scales applications as necessary.
Monitoring and Logging: Various tools can be integrated to monitor and log cluster activity, ensuring optimal performance and health.

Complete Steps to Set Up a Kubernetes Cluster Using Kubeadm