(Day 38) Task : Mastering Kubernetes Jobs in the Container and Pod Lifecycle :-

Introduction :-

Kubernetes has revolutionised how modern applications are deployed, scaled, and managed. While most developers and DevOps engineers are familiar with Deployments and StatefulSets to manage long-running services, one of the most important and powerful features of Kubernetes for one-time or batch processing tasks is the Job.

A Kubernetes Job ensures that a pod successfully completes the assigned task, even if it requires multiple attempts. Combined with CronJobs and Init Containers, Jobs become a key part of automating routine tasks, batch processing, and managing the container and pod lifecycle effectively.

In this article, we will go deep into understanding Kubernetes Jobs, their use cases, associated YAML configurations, how they behave in the pod lifecycle, how to schedule them using CronJobs, and how Init Containers can prepare your main containers for execution. We’ll also include real-world examples, YAML configurations, installation instructions using Minikube, and hands-on commands to help you master Jobs in Kubernetes.

Setting Up Your Environment :-

To follow along with the examples, you’ll need a working Kubernetes cluster. We’ll use Minikube for local development and testing.

Prerequisites:

• Linux/macOS or EC2 instance.

• Docker.

• kubectl.

• Minikube.

Step 1: Install Docker :

sudo apt update
sudo apt install docker.io -y 
sudo systemctl start docker 
sudo systemctl enable docker 
sudo usermod -aG docker $USER 
newgrp docker

Test Docker :

docker run hello-world

Step 2: Install Minikube :

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube

Start your cluster :

minikube start --driver=docker

Verify :

kubectl get nodes

What is a Kubernetes Job?

A Job creates one or more pods to perform a specific task to completion. Unlike a Deployment, which maintains a certain number of pods running continuously, a Job runs a pod until it completes successfully (exit code 0), and then it finishes.

Use Cases for Kubernetes Jobs:

• Database migration scripts

• Scheduled backups

• Email notifications

• Temporary data cleanup

• Batch processing pipelines

Basic Job YAML Example:

apiVersion: batch/v1
kind: Job
metadata:
  name: testjob
spec:
  template:
    spec:
      containers:
        - name: testjob
          image: centos:7
          command: ["/bin/sh", "-c", "echo Technical-Guftgu; sleep5"]
      restartPolicy: Never

Apply :

kubectl apply -f job.yaml

Check Job :

kubectl get jobs
kubectl get pods
watch kubectl get pods

Parallel Jobs in Kubernetes :-

Kubernetes Jobs also support parallelism, which allows you to run multiple pods at the same time. This is useful when you have batch tasks that can be split and processed concurrently.

YAML Example with Parallelism :

apiVersion: batch/v1
kind: Job
metadata:
  name: testjob
spec:
  parallelism: 5                 # Run 5 pods in parallel
  activeDeadlineSeconds: 30     # Timeout after 30 seconds
  template:
    metadata:
      name: testjob
    spec:
      containers:
        - name: counter
          image: centos:7
          command: ["/bin/bash", "-c", "echo Technical-Guftgu; sleep 20"]
      restartPolicy: Never

Apply it and check :

kubectl apply -f job2.yml
kubectl get jobs
kubectl get pods

CronJobs :-

A CronJob in Kubernetes allows you to run Jobs on a scheduled basis, just like Linux cron. The format is: minute hour day month day-of-week.

CronJob YAML Example :

apiVersion: batch/v1
kind: CronJob
metadata:
  name: bhupi
spec:
  schedule: "* * * * *"  # Runs every minute
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: bhupi
              image: ubuntu
              command: ["/bin/bash", "-c", "echo Technical-Guftgu; sleep 5"]
          restartPolicy: Never

Apply and monitor :

kubectl apply -f cronjob.yaml
kubectl get cronjobs
watch kubectl get jobs

View logs :

kubectl get pods
kubectl logs <pod-name>

Init Containers in Jobs :-

Init Containers are special containers that run before the main containers start. They are great for setup tasks like creating files, downloading dependencies, or setting environment variables.

Init Container with Shared Volume Example :

apiVersion: v1
kind: Pod
metadata:
  name: init-container-demo
spec:
  volumes:
    - name: exchange
      emptyDir: {}
  initContainers:
    - name: c1
      image: ubuntu
      command: ["/bin/bash", "-c", "echo 'Hello Aditya – self3' > /tmp/exchange/testfile"]
      volumeMounts:
        - name: exchange
          mountPath: /tmp/exchange
  containers:
    - name: c2
      image: ubuntu
      command: ["/bin/bash", "-c", "while true; do echo 'Reading shared file:'; cat /tmp/data/testfile; sleep 5; done"]
      volumeMounts:
        - name: exchange
          mountPath: /tmp/data

Apply :

kubectl apply -f init-container-demo.yaml
kubectl logs -f init-container-demo

Understanding Pod & Container Lifecycle with Jobs :-

Kubernetes jobs deeply connect with the pod lifecycle:

1. Pod Creation: Triggered by the Job controller.

2. Init Containers: Run in order, must complete first.

3. Main Containers: Start after init containers finish.

4. Completion: Pod exits with 0 (success).

5. Job Controller: Marks job as Complete.

Restart Policies:

• Never: Job doesn’t retry on failure.

• OnFailure: Retry on non-zero exit code (default in some controllers).

Best Practices :