Static Pods, Manual Scheduling, Labels, and Selectors in Kubernetes

Introduction

In Kubernetes, effective resource management and organization are crucial for maintaining a robust and scalable environment. Labels and selectors play a vital role in achieving this by allowing you to categorize and filter Kubernetes objects such as pods, services, and deployments based on meaningful criteria. In this blog, we will explore how to use labels and selectors to manage Kubernetes resources efficiently. We’ll also delve into related concepts like namespaces, annotations, static pods, and manual pod scheduling, providing you with practical insights and hands-on tasks to deepen your understanding.

---

📌 Labels and Selectors in Kubernetes

Labels 🏷️

Labels are key-value pairs attached to Kubernetes objects like pods, services, and deployments. They help organize and group resources based on criteria that make sense to you.

Examples of Labels:

• environment: production

• type: backend

• tier: frontend

• application: my-app

Selectors 🔍

Selectors filter Kubernetes objects based on their labels, making it easy to query and manage a subset of objects that meet specific criteria.

Common Usage:

• Pods: kubectl get pods --selector app=my-app

• Deployments: Filter the pods managed by a deployment.

• Services: Filter the pods to which the service routes traffic.

---

Labels vs. Namespaces 🌍

• Labels: Organize resources within the same or across namespaces.

• Namespaces: Provide a way to isolate resources from each other within a cluster.

Annotations 📝

Annotations are similar to labels but attach non-identifying metadata to objects. For example, they can be used to record the release version of an application or details of the last applied configuration.

---

🛠️ Static Pods

Static Pods are special types of pods managed directly by the kubelet on each node rather than through the Kubernetes API server.

Key Characteristics of Static Pods:

• Not Managed by the Scheduler: Unlike deployments or ReplicaSets, the Kubernetes scheduler does not manage static pods.

• Defined on the Node: Configuration files for static pods are placed directly on the node's file system, and the kubelet watches these files.

Examples of Static Pods:

• ApiServer

• Kube-scheduler

• Controller-manager

• ETCD

Managing Static Pods:

• SSH into the Node: Gain access to the node where the static pod is defined (usually the control plane node).

• Modify the YAML File: Edit or create the YAML configuration file for the static pod.

• Remove the Scheduler YAML: To stop the pod, remove or modify the corresponding file directly on the node.

• Default Location: The directory is usually /etc/kubernetes/manifests/, where you can place the pod YAML file for the kubelet to schedule it.

---

🧭 Manual Pod Scheduling

Manual scheduling in Kubernetes involves assigning a pod to a specific node rather than letting the scheduler decide.

Key Points:

• nodeName Field: Use this field in the pod specification to specify the node where the pod should run.

• No Scheduler Involvement: When nodeName is specified, the scheduler bypasses the pod, and it’s directly assigned to the given node.

---

Task 1: Create a Pod and Schedule It Manually Without the Scheduler

To manually schedule a pod without using the Kubernetes scheduler, you can achieve this by specifying the nodeName field in the Pod's YAML configuration. By setting the nodeName, you're directly telling Kubernetes on which node the pod should run, bypassing the default scheduling process.

Task Steps:

1. Identify the Node Name:

   • First, get the name of the node where you want to manually schedule the pod:

       kubectl get nodes
     

2. Create the Pod YAML File:

   • Create a YAML file with the nodeName field specified. Here's an example YAML configuration:

       apiVersion: v1
       kind: Pod
       metadata:
         name: manually-scheduled-pod
       spec:
         containers:
         - name: nginx-container
           image: nginx
         nodeName: <your-node-name>
     

Replace <your-node-name> with the actual node name you obtained from the kubectl get nodes command.

3. Apply the Pod YAML:

   • Once your YAML file is ready, create the pod using:

       kubectl apply -f <your-pod-file>.yaml
     

4. Verify the Pod Placement:

   • Check if the pod has been scheduled on the specified node:

       kubectl get pods -o wide
     

This command will display detailed information about the pod, including the node on which it is running.

Explanation:

• nodeName: This field in the pod spec tells Kubernetes exactly which node to run the pod on, bypassing the Kubernetes scheduler.

• Manual Scheduling: By specifying nodeName, you're manually scheduling the pod, which can be useful for specific scenarios like testing, debugging, or ensuring a pod runs on a specific node.

---

Task 2: Restart the Control Plane Components

2.1 Restart Control Plane Components

1. Find the Directory: Make sure you are in the /etc/kubernetes/manifests directory where the control plane component manifests are located.

2. Restart Control Plane Components: You can restart the components by deleting their manifests or by making a change in them.

    # Example: to restart the kube-apiserver
    mv kube-apiserver.yaml kube-apiserver.yaml.bak
    sleep 5
    mv kube-apiserver.yaml.bak kube-apiserver.yaml
   

3. Verify the Restart: Check the status of the control plane components.

    kubectl get pods -n kube-system
   

---

Task 3: Create 3 pods with the name as pod1, pod2 and pod3 based on the nginx image and use labels as env:test, env:dev and env:prod for each of these pods respectively.

Then using the kubectl commands, filter the pods that have labels dev and prod.

Task Steps:

1. Create the Pods:

   • First, create three YAML files for the pods named pod1.yaml, pod2.yaml, and pod3.yaml with the specified labels.

pod1.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: pod1
      labels:
        env: test
    spec:
      containers:
      - name: nginx
        image: nginx

pod2.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: pod2
      labels:
        env: dev
    spec:
      containers:
      - name: nginx
        image: nginx

pod3.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: pod3
      labels:
        env: prod
    spec:
      containers:
      - name: nginx
        image: nginx

2. Apply the YAML Files:

   • Use the following kubectl command to create the pods:

       kubectl apply -f pod1.yaml
       kubectl apply -f pod2.yaml
       kubectl apply -f pod3.yaml
     

3. Filter Pods Based on Labels:

   • To filter and display pods with the labels dev and prod, use the following kubectl command:

       kubectl get pods --selector=env=dev
       kubectl get pods --selector=env=prod
     

   • Alternatively, you can filter both dev and prod labels in a single command:

       kubectl get pods --selector='env in (dev,prod)'
     

Explanation:

• Pod Creation: The YAML files define the pods with specific labels (env: test, env: dev, and env: prod). The nginx image is used for all three pods.

• Filtering Pods: The kubectl get pods --selector=env=<value> command is used to filter pods based on their labels. Using --selector='env in (dev,prod)' allows filtering multiple labels in one command.

---

Conclusion

Understanding and utilizing labels and selectors in Kubernetes can significantly enhance your ability to manage and query resources across your clusters. By mastering these concepts, along with related topics such as namespaces, annotations, static pods, and manual scheduling, you’ll be well-equipped to handle complex Kubernetes deployments. Whether you’re filtering pods by environment or scheduling specific workloads on designated nodes, these techniques will empower you to maintain an organized and efficient Kubernetes environment.

---

References

Visual Insight: Day 13 Static Pods, Manual Scheduling, Labels, and Selectors in Kubernetes