Day 35 of 90 Days of DevOps Challenge: Autoscaling in kubernetes

In Day 34, I explored how Kubernetes Deployments and ReplicaSets manage pods and ensure high availability through manual scaling. While that gave me insight into how Kubernetes handles application workloads, I realized that manually adjusting replicas isn't scalable in real-world environments with fluctuating traffic.

So today, on Day 35, I dove into Autoscaling in Kubernetes, a smarter way to handle resource demands automatically. I explored different types of autoscaling, with a focus on the Horizontal Pod Autoscaler (HPA). This built-in mechanism allows Kubernetes to scale pods up or down based on CPU or memory usage, helping maintain performance and efficiency with minimal manual intervention.

What is Autoscaling?

Autoscaling is the automatic process of increasing or decreasing infrastructure resources (like servers or containers) based on demand.

Whether your application is facing a sudden traffic spike or a low-usage period, autoscaling ensures:

• More resources are added when the load increases

• Unused resources are removed during idle times

Types of Autoscaling

Autoscaling generally happens in two forms:

1. Horizontal Scaling

• Involves adding or removing instances (e.g., pods, VMs, containers).

• Preferred in cloud-native environments like Kubernetes.

• Common Example: Auto-scaling number of pods in a Deployment.

2. Vertical Scaling

• Involves increasing the capacity (CPU/RAM) of a single instance.

• Riskier in production due to potential downtime.

• Example: Increasing CPU of a running VM.

Kubernetes Autoscaling Mechanisms

Kubernetes supports two built-in types of autoscaling:

1. Horizontal Pod Autoscaler (HPA)

HPA scales the number of pods up or down based on observed metrics like CPU or memory usage. It’s the most commonly used approach, ideal for managing traffic spikes in stateless applications.

2. Vertical Pod Autoscaler (VPA).

VPA, on the other hand, adjusts the CPU and RAM requests/limits of a pod dynamically. While it helps optimize individual pod performance, it's rarely used in production due to the need to restart pods when applying changes.

Add-ons or Extensions to Kubernetes:

3. Cluster Autoscaler (CA)
Cluster Autoscaler scales the number of nodes in your cluster. When there are not enough resources (like CPU/RAM) to schedule new pods, CA adds nodes; when nodes are underutilized, it removes them. It’s especially useful in cloud environments (EKS, GKE, AKS), but needs to be installed separately as an official Kubernetes add-on.

4. Custom Pod Autoscaler (CPA)
CPA allows users to define custom logic for autoscaling based on external or custom metrics (e.g., queue length, request rate, etc.). It is not built-in and typically implemented using custom controllers or tools like KEDA. CPA is powerful for event-driven or specialized applications that require non-standard scaling policies.

Horizontal Pod Autoscaler (HPA)

What is HPA?

HPA (Horizontal Pod Autoscaler) automatically scales the number of pod replicas in a deployment, replica set, or stateful set based on observed metrics like:

• CPU utilization

• Memory usage

• Custom metrics (with extensions)

How it Works:

• HPA continuously monitors metrics such as CPU and memory utilization (or even custom metrics).

• Based on the defined thresholds in the HPA configuration, it decides whether to scale up (add more pod replicas) or scale down (remove pod replicas).

• The scaling decision is based on target utilization versus current utilization.

For example, if CPU usage is consistently above 80%, HPA may increase the number of pods to distribute the load evenly.

Integration with Metrics Server

• HPA relies on a component called the Metrics Server to gather real-time performance data from pods and nodes.

• Without the Metrics Server installed, HPA will not function, as it cannot access resource usage data.

1. HPA monitors resource usage of pods via Metrics Server.

2. Based on target utilization thresholds, it calculates the desired number of replicas.

3. It updates the deployment to match the new replica count.

Understanding Metrics Server

Metrics Server is a lightweight aggregator of resource usage data in Kubernetes.

It collects metrics such as:

• CPU and memory utilization of pods and nodes

• These metrics are crucial for HPA to function

Check Available Metrics:

# View node metrics
$ kubectl top nodes

# View pod metrics
$ kubectl top pods

NOTE: Metrics Server is not deployed by default in Kubernetes clusters. You must install it manually to enable HPA functionality.

Vertical Pod Autoscaler

While Horizontal Pod Autoscaler (HPA) adjusts the number of pod replicas, Vertical Pod Autoscaler (VPA) focuses on a single pod’s resource allocation. It automatically updates a pod’s CPU and memory (RAM) requests and limits based on its actual usage over time.

How VPA Works

• VPA monitors the historical and current resource usage of a pod.

• Based on the collected data, it recommends or applies updated resource values to ensure that the pod runs efficiently.

• When a change is needed, the pod is recreated (terminated and restarted) with new resource values.

• It works best for stateful applications or workloads with predictable load patterns.

VPA Components

1. Recommender: Analyzes resource usage and suggests optimal CPU/memory values.

2. Updater: Applies recommendations by restarting pods when needed.

3. Admission Controller: Sets recommended values during pod (re)creation.

Why VPA is Rarely Used

• Pod Restart Required: Changes are applied only after restarting the pod, which may affect availability.

• Not Compatible with HPA: VPA and HPA cannot modify the same resource (CPU) simultaneously, this can lead to conflicts.

• Lack of Fine-grained Control: It's more suited for batch jobs or infrequently updated workloads than highly dynamic systems.

VPA Use Cases

• Batch jobs with varying memory/CPU demands.

• Stateful workloads where pod restarts are acceptable.

• Optimizing pods in non-production or low-availability environments.

Final Thoughts

Autoscaling is vital in modern DevOps and Kubernetes workflows. While both vertical and horizontal scaling have their places, HPA is the go-to tool for responsive, production-grade scalability.

• Horizontal Scaling → Fast, resilient, and dynamic

• Vertical Scaling → Limited use, potential downtime

• Metrics Server → Core dependency for resource-aware autoscaling

Whether you're building microservices or deploying large-scale applications, understanding and implementing HPA is key to reliability and performance.