Kubernetes Networking : Day 32 of 40daysofkubernetes

Introduction

Networking is a fundamental aspect of Kubernetes, though it can be complex to understand fully. Kubernetes addresses four main networking challenges:

1. Container-to-Container Communication: Managed by Pods, which enable containers within the same Pod to communicate using localhost.

2. Pod-to-Pod Communication: Focuses on how Pods interact with each other across the cluster.

3. Pod-to-Service Communication: Services handle how Pods connect with external services and applications.

4. External-to-Service Communication: Deals with how external systems access applications running within Kubernetes Services.

Understanding these networking aspects is crucial for ensuring smooth and effective communication within your Kubernetes environment. In this blog, we’ll explore each of these categories in detail to help you grasp how Kubernetes manages networking. Let’s dive in!

Container-to-Container Networking

Container-to-container networking occurs within a Pod’s own network space. Each Pod has a separate network setup that isolates its containers from the rest of the system. Containers within the same Pod share the same IP address and communicate with each other using localhost. This means they operate as if they are on the same machine, thanks to being in the same network space (illustrated by the green line in the diagram).

Pod-to-Pod Networking

In Kubernetes, each node is assigned a range of IP addresses (CIDR) for Pods. This ensures that each Pod gets a unique IP address that is visible to other Pods in the cluster. Unlike container-to-container communication, which uses localhost, Pod-to-Pod communication relies on these unique IP addresses.

When a Pod communicates with another Pod, whether on the same node or different nodes, the traffic must move between the Pod's network space and the node’s main network space. This is achieved using a virtual ethernet device called a veth pair. Each Pod's network space is connected to the main network space through these veth pairs.

Here’s how it works:

• Traffic from Pod 1: It travels through its network interface eth0 to a virtual interface veth0 in the node’s main network space.

• Virtual Bridge: The traffic moves through a virtual network bridge that connects veth0 to veth1.

• Traffic to Pod 2: Finally, the traffic travels through veth1 and reaches Pod 2’s eth0 interface.

The Container Network Interface (CNI) is a set of standards and plugins that manage network configuration for Pods. It ensures each Pod gets its IP address and handles connectivity between Pods. CNI plugins create virtual interfaces and bridges, allowing Pods to communicate effectively.

Example: Communication Between Two Pods on the Same Node

Let’s consider two Pods, Pod A and Pod B, on the same node:

• Network Setup:

  • Pod A has a veth pair: vethA (in the Pod) and vethA' (on the host).

  • Pod B has a veth pair: vethB (in the Pod) and vethB' (on the host).

  • vethA' and vethB' are connected to the bridge interface cni0 on the host.

• Communication:

  • Pod A sends a packet from its eth0 interface.

  • The packet travels through vethA to vethA', reaches the cni0 bridge on the host, and is then forwarded to vethB'.

  • The packet enters Pod B through its eth0 interface.

Example: Communication Between Two Pods on Different Nodes

For Pods on different nodes, the process is similar but includes additional routing steps:

• Routing Between Nodes:

  • Pod A sends a packet via eth0.

  • The packet travels through the veth pair, reaches the cni0 bridge on Node 1, and is routed to Node 2 based on Pod B’s IP address.

  • On Node 2, the packet enters the cni0 bridge, travels through the veth pair, and reaches Pod B’s eth0 interface.

Pod-to-Service Networking

Pods in Kubernetes can change frequently, making it challenging to keep track of their IP addresses. Kubernetes uses Services to address this issue:

• Static Virtual IP: A Service provides a stable IP address that clients use to connect to Pods, even if a Pod’s IP changes.

• Load Balancing: The Service distributes incoming traffic to the appropriate backend Pods.

• Dynamic Management: The Service tracks available Pods, so clients always connect through the Service’s static IP.

Kubernetes uses two methods for load balancing within the cluster:

• IPTABLES: kube-proxy sets up rules to redirect traffic to the correct Pod. This method is efficient and has low overhead.

• IPVS: A more advanced method that offers faster and more reliable traffic management by operating directly in the kernel.

For detailed information on how Services work and interact, check out my blog.

External-to-Service Networking

To expose an application to external networks, you can use two main methods:

• Egress: Routes traffic from Kubernetes Pods to the Internet. Traffic appears to come from the node, not the Pod itself, due to source NAT performed by iptables.

• Ingress: Manages incoming traffic from the outside world into Kubernetes Services. It also controls which connections are allowed or blocked using rules. Common ingress solutions include:

  • Service Load Balancer: Distributes incoming traffic to the Pods.

  • Ingress Controller: Manages routing and rules for external access to Services.

These methods help manage how traffic flows in and out of your Kubernetes cluster. We will explore Ingress Controllers in more detail in our next blog.

Conclusion

Kubernetes networking is made possible by the Container Network Interface (CNI), which simplifies network configuration. Each Pod gets a unique IP address, and CNI plugins ensure Pods can communicate across the cluster, whether on the same or different nodes. CNI plugins like Cilium, Calico, and others implement the required networking model, making Kubernetes networking seamless and scalable. Network policies can be used to control and secure communication between Pods.

Resources I used