🚀 Day-05: DaemonSets in Kubernetes

🛠What is a DaemonSet in Kubernetes?

A DaemonSet is a Kubernetes object that ensures a specific pod runs on all (or selected) nodes in a Kubernetes cluster. When you create a DaemonSet, Kubernetes will automatically deploy a copy of the pod to every node, ensuring that each node runs one instance of the pod.

🤔 Why Use a DaemonSet?

DaemonSets are ideal for running background services or tasks across all nodes in a cluster. These tasks are typically system-level or monitoring applications that need to operate on every node to collect data or perform maintenance.

🎯Purpose of DaemonSet

1. 🔍Node Monitoring: Deploying monitoring agents like Prometheus Node Exporter or Datadog agent on every node to collect metrics.

2. 📝Logging: Running log collection agents (like Fluentd or Filebeat) on all nodes to gather and forward logs.

3. 🌐Networking: Setting up network services like VPNs, CNI plugins, or other networking tools that must run on every node.

4. 🔒Security: Deploying security agents for tasks like vulnerability scanning or runtime protection on each node.

5. 🧹System Maintenance: Running background tasks or services that manage or clean resources on each node.

🧠 Key Points to Remember

• A DaemonSet ensures that a pod is always running on all eligible nodes.

• If a new node is added to the cluster, the DaemonSet automatically deploys the required pod to the new node.

• If a node is removed, the pod on that node is automatically cleaned up.

• DaemonSets can be used with node selectors, tolerations, and affinities to target specific nodes.

• 

📝 Creating a DaemonSet: Step-by-Step Guide

Create the DaemonSet YAML Definition:

vi ds-pod.yaml

Apply the YAML Definition:

kubectl apply -f ds-pod.yaml

Check Available DaemonSets:

kubectl get ds

Verify Pods for DaemonSet:

kubectl get pods -o wide

Scale the Cluster and Verify:

• Increase the worker node count to 3 using the AutoScaling Group on the AWS Management Console.

• Check the pod count again; a new pod will automatically be deployed to the new node.

kubectl get ds
kubectl get pods -o wide

Cleanup the DaemonSet:

kubectl delete -f ds-pod.yaml

Verify Pods After Cleanup:

kubectl get pods

🎯What We Aim to Achieve with DaemonSets

• Create a DaemonSet: Define a DaemonSet using a YAML file (ds-pod.yaml) that specifies the pod to deploy on each node.

• Deployment and Verification: Apply the DaemonSet and verify that a pod is running on every node.

• Scaling the Cluster: Show how Kubernetes automatically deploys the pod on new nodes when the cluster is scaled up.

• Clean Up: Demonstrate how to delete the DaemonSet and remove the pods from all nodes.

🌍 Real-World Examples of DaemonSets

Example 1: Monitoring Nodes with Prometheus Node Exporter

• Scenario: Imagine you’re running a Kubernetes cluster with multiple nodes, and you need to monitor the health and performance of each one.

• Solution: You can deploy the Prometheus Node Exporter as a DaemonSet. This ensures that every node in your cluster has a monitoring agent running, collecting important system metrics like CPU and memory usage.

Example 2: Collecting Logs with Fluentd

• Scenario: You need to collect logs from all the applications running in your Kubernetes cluster and send them to a centralized logging system.

• Solution: Deploy Fluentd as a DaemonSet. This setup makes sure that each node is collecting logs and forwarding them to your chosen logging service, helping you keep track of everything happening in your cluster.

🛠 Troubleshooting DaemonSets

When you notice that your DaemonSet isn't working as expected, here’s how to quickly diagnose and fix the issues.

Common Signs of Problems

• Pods Not Running: Some or all pods aren’t in the Running state.

• CrashLoopBackOff: Pods keep restarting due to errors.

• ImagePullBackOff: Pods can’t pull the container image from the registry.

• Pending Pods: Pods are stuck and not starting.

Easy Troubleshooting Steps

1. Check DaemonSet Status

   • Run this command to see detailed information about your DaemonSet:

    kubectl describe daemonset <daemonset-name>

• Look for any error messages or events that indicate issues.

2. Inspect Pod Status

   • Check the status of all pods created by the DaemonSet:

    kubectl get pods -o wide -l name=<daemonset-name>

• This will help you identify which pods are having problems.

3. View Pod Logs

   • For any pods that are failing, view their logs to find error messages:

    kubectl logs <pod-name>

• Logs can reveal what’s going wrong inside the application.

4. Check Node Resources

   • Ensure there are enough resources (CPU, memory) on the nodes:

    kubectl describe node <node-name>

• If resources are low, consider scaling up your nodes or adjusting resource requests in the DaemonSet.

5. Image Availability

   • Verify that the image used in the DaemonSet is correct and accessible. You can test pulling the image manually to check for issues:

    docker pull <image-name>

6. Node Health

   • Check if the nodes are healthy. A node in a NotReady state might cause issues:

    kubectl get nodes

Final Tips

• Always check the events and logs for specific error messages; they provide valuable clues.

• If you need to make changes, update the DaemonSet configuration and apply it again.

✅ Conclusion

In this task, we explored how to effectively use DaemonSets in Kubernetes. By creating a DaemonSet, we ensured that a specific application runs on all nodes in our cluster, providing essential services like monitoring and logging. We saw how Kubernetes automatically scales our application when new nodes are added and how easy it is to clean up when no longer needed. Understanding DaemonSets is crucial for managing system-level applications efficiently, ensuring that our infrastructure remains consistent and reliable.

Stay tuned for more insights on Kubernetes features and best practices in our upcoming posts!🥳