Kubernetes Networking & Multi-Tenant Guide

Introducing of Kubernetes’s Network

Kubernetes, as we know, is a powerful platform for automating the deployment, scaling, and operations of application containers across clusters of hosts. However, for these containers, or rather 'Pods' as they are known within the Kubernetes realm, to effectively communicate and serve their purpose, a robust networking model is essential. Let's delve into the components that constitute networking in Kubernetes.

Firstly, we have "Communication between Pods on the same Node." This is the most basic level of networking within Kubernetes. Pods running on the same physical or virtual machine need to communicate with each other. Kubernetes uses the networking namespace and other isolation mechanisms provided by the underlying operating system to ensure that these Pods can connect efficiently and securely.

Moving a step further, we encounter the scenario of "Communication between Pods on different Nodes." As our applications scale, they span across multiple Nodes, creating a need for cross-node communication. Kubernetes abstracts the complexity involved in this process, ensuring Pods can communicate seamlessly, irrespective of their physical location within the cluster.

Next, let's talk about "Service." A Service in Kubernetes is an abstraction that defines a logical set of Pods and a policy by which to access them. This abstraction enables the decoupling of backend Pod implementations from the frontend services that access them. Services allow for stable, reliable communication pathways between different components of an application.

"Ingress and Egress" mechanisms in Kubernetes facilitate the management of incoming and outgoing traffic to your cluster. Ingress controls how external traffic reaches your services within the cluster, allowing you to define accessible URLs, load balance traffic, terminate SSL, and offer name-based virtual hosting. Egress, on the other hand, controls the outbound traffic from your Pods to the external world, ensuring that only authorized connections are made, thus enhancing the security of your cluster. Although Kubernetes does not have native Egress resources, some other components provide this implementation, such as Calico.

Lastly, we have "NetworkPolicy." This is a specification of how groups of Pods are allowed to communicate with each other and other network endpoints. NetworkPolicies are crucial for enforcing security rules and ensuring that only the intended traffic can flow between Pods, Services, and the external world.

These are the basic concepts that will be encountered when discussing Kubernetes networking. However, these concepts require some specific components to implement their functions.

Kubernetes Networking Components

The following are some of the main components.

CNI Implementations:

Let's start with the foundation of Kubernetes networking - the Container Network Interface, or CNI. The CNI is a specification and a set of tools that facilitate the configuration of network interfaces for containers. Kubernetes relies on CNI implementations to provide in-pod networking capabilities. These implementations, such as Calico, Flannel, and Cilium, to name a few, allow for a pluggable framework that supports a wide range of networking features including network segmentation, policy enforcement, and overlay networks. Choosing the right CNI implementation is critical as it influences the network performance, security, and scalability of your Kubernetes cluster.

For example, A few years ago, Flannel, which was widely used, did not provide network policy support. Therefore, if this CNI plugin is used, network policies cannot take effect.

However, in addition to the native NetworkPolicy of Kubernetes, Calico and Cilium have also implemented their own enhanced versions of network policy.

kube-proxy:

Moving on, we encounter kube-proxy, a vital component that runs on each node in the Kubernetes cluster. kube-proxy is responsible for maintaining network rules on nodes. These rules allow network communication to your Pods from network sessions inside or outside of your cluster. Essentially, kube-proxy enables the forwarding of TCP and UDP packets to pods based on the IP and port number of the incoming request. Through services, kube-proxy abstracts the IP addresses of individual pods, ensuring that the internal network structure is hidden from the external clients.

DNS:

In the realm of Kubernetes, DNS plays a pivotal role in service discovery. It allows Pods to locate each other and external services through human-readable names. When a Kubernetes cluster is created, a DNS service is automatically deployed, assigning DNS names to other Kubernetes services. This simplifies communication within the cluster, as Pods can reach each other and external resources using these DNS names, enhancing the overall application architecture's flexibility and robustness.

Controller:

Lastly, let's delve into the controller, particularly focusing on IPAM, and the Endpoint/EndpointSlice/Service mechanisms.

IPAM (IP Address Management): This functionality is crucial for managing the assignment of IP addresses to Pods and Services. Efficient IPAM practices ensure that there is no conflict between the IPs assigned within the cluster, maintaining a smooth network operation.
Endpoint/EndpointSlice/Service: These components work together to manage how external and internal communications reach the Pods. A Service defines a logical set of Pods and a policy by which to access them. The Endpoints and EndpointSlices keep track of the IP addresses of the Pods that are associated with each Service. This system ensures that network traffic is efficiently routed to the appropriate Pods, allowing for scalable and reliable application deployments.

Although they may appear complex, they all follow the most basic models.

Kubernetes Networking Model

The kubernetes networking model is ingeniously designed to ensure seamless communication within the cluster, which is pivotal for deploying scalable and resilient applications. Let's explore the core principles that define this networking model.

Each Pod has its own IP address:

The first principle to understand is that in Kubernetes, each Pod is assigned its own unique IP address. This is a significant departure from traditional deployments where containers or applications within a host might share the host's IP address. This unique IP per Pod simplifies application configuration and inter-service communication. It means that each Pod can be treated as if it's a physical host on the network, making network policies and communications more straightforward to manage.

All Pods can communicate across Nodes without the need for NAT

Components on the Node (e.g., kubelet) can communicate with all Pods on the Node:

These two rules have been introduced earlier, and they define how Pods are accessed.

How to Access Applications Deployed in Kubernetes

Let’s discuss How to Access Applications Deployed in Kubernetes

Deploying applications is only part of the journey. An equally crucial aspect is how these applications, once deployed, are made accessible to the outside world.

NodePort:

Let's start with NodePort, a simple yet powerful way to access your applications. When you configure a service as NodePort, Kubernetes exposes that service on each Node’s IP at a static port (the NodePort). A NodePort service is accessible by the IP of the Node it's running on combined with a high port number assigned specifically for that service. This means that anyone who can reach the Node, either within the internal network or from the outside world, can access the service by hitting the Node's IP at the designated port.

This method is particularly useful for development environments or smaller setups where direct access to each node is manageable. However, it's worth noting that managing access through NodePort can become challenging in environments with a large number of Nodes, as it requires keeping track of multiple IP addresses and ports.

LoadBalancer:

Moving on to a more sophisticated and widely used approach – LoadBalancer. This method integrates Kubernetes services with existing cloud providers' load balancers. When you create a service of type LoadBalancer, Kubernetes provisions a cloud load balancer for your service, and directs external traffic to it. This external load balancer then automatically routes the traffic to your Kubernetes Pods, regardless of which Node they're running on.

The beauty of using LoadBalancer is its simplicity and scalability. It abstracts away the complexity of dealing with individual Node IPs and ports, providing a single entry point – the load balancer's IP – to access your service. This makes it an ideal choice for production environments where reliability and scalability are paramount.

Whether you opt for NodePort for its simplicity and direct access, or LoadBalancer for its robustness and scalability, Kubernetes offers flexible solutions to suit different application access needs.

But you may be curious, in most cases we talk about accessing applications inside the cluster through Ingress from outside the cluster. What exactly is Ingress?

What is Ingress

Ingress: A Specification

At its core, Ingress is not merely a tool or a component; it is a powerful API object that provides a sophisticated method for handling external access to the services in a Kubernetes cluster. It allows you to define flexible, HTTP-based routing rules that direct traffic to different services based on the request's details. Let's delve into the key elements that make up an Ingress specification:

Host:

The 'host' field in an Ingress specification is what allows us to define domain names that will be routed to specific services within our cluster. This means that you can have different domain names or subdomains pointing to different parts of your application, all managed through a single Ingress resource. This is particularly useful for applications that require multiple entry points or for hosting multiple services under the same cluster.

Paths:

Next, we have 'paths.' This element works in conjunction with 'host' to provide even finer control over the routing of incoming requests. By specifying paths, you can direct traffic not just based on the domain name, but also based on the URL path. For instance, requests to example.com/api could be routed to one service, while requests to example.com/blog could be directed to another. This allows for a highly customizable and efficient distribution of traffic to the various components of your applications.

Certificates:

Lastly, an essential aspect of modern web applications is security, particularly the use of SSL/TLS certificates to encrypt traffic. Ingress facilitates this by allowing you to specify certificates for each host, ensuring that all communication is securely encrypted. This integration of SSL/TLS certificates with Ingress means that you can manage the security of your services at the same point where you're managing their accessibility, simplifying the overall configuration and maintenance of your applications.

However, simply creating an Ingress resource is not enough for it to work.

How does Ingress work?

The Ingress Controller: Translating Rules into Action

At the heart of the Ingress mechanism lies the Ingress controller. This isn't just any controller; it's the maestro of network traffic, orchestrating how requests from outside the Kubernetes cluster are handled and directed. But how does it achieve this?

The Ingress controller continuously monitors the Kubernetes API for Ingress resources - the rules and paths that you define for routing external traffic. Upon detecting an Ingress resource, the controller springs into action, translating these high-level Ingress rules into specific, actionable dataplane rules. This translation process is where the abstract becomes concrete, turning our defined paths and domains into precise instructions on how traffic should flow.

The Dataplane: Carrying Traffic to its Destination

With the rules translated, we now turn to the dataplane - the physical and logical network paths that actual traffic flows through. The dataplane is where the rubber meets the road, or more aptly, where the packet meets the pod.

In the context of Ingress, the dataplane is responsible for carrying the external traffic, now governed by the rules set forth by the Ingress controller. This can involve a variety of operations, such as SSL termination, load balancing, and content-based routing, all performed with the goal of ensuring that incoming requests are delivered to the correct service within the cluster.

Usually, we create a service for the dataplane and then expose it to external access by using NodePort or LoadBalancer as mentioned earlier.

Limitations of Ingress

Limited Expressiveness

One of the core limitations of Ingress lies in its limited expressiveness. Ingress rules are designed to be straightforward, focusing primarily on HTTP and HTTPS routing. This simplicity, while beneficial for ease of use and understanding, means that Ingress cannot natively handle more complex routing scenarios. For example, Ingress does not inherently support advanced load balancing algorithms, TCP or UDP traffic routing, or canary deployments out of the box.

This limitation can pose challenges for applications requiring more sophisticated traffic management and routing capabilities.

Reliance on Annotations for Controller Implementation Extensions

Another significant limitation is the way Ingress extends its functionality - through annotations. Each Ingress controller, such as Ingress-NGINX, Kong Ingress controller, or HAProxy, may implement additional features that are not part of the standard Ingress specification. These features are often enabled or configured via annotations in the Ingress resource.

While annotations provide a flexible way to extend Ingress capabilities, they also introduce variability and complexity. Different Ingress controllers might support different sets of annotations, leading to a lack of standardization across implementations. This can result in portability issues when moving applications between clusters using different Ingress controllers. Furthermore, relying heavily on annotations can make Ingress resources cluttered and harder to manage, especially as the number of annotations grows to accommodate more complex configurations.

Advantages of Gateway API

Considering these limitations of Ingress, the Kubernetes community decided to make some changes and thus began the design of the Gateway API.

Gateway API offers several key advantages that make it an attractive choice for managing network traffic in Kubernetes:

Role-oriented:

Gateway API is designed with distinct roles for different types of users. This allows for a more fine-grained and role-specific way of controlling access to resources, improving security, and making the management of permissions more straightforward. For example, cluster operators can control infrastructure-related aspects of gateways and routes, while developers can manage application-level configurations.

More Universal:

Unlike Ingress, which primarily focuses on HTTP/HTTPS traffic, Gateway API is protocol-agnostic and can handle a broader range of traffic types, including TCP and UDP. This makes it a more universal solution for managing all types of network traffic within a Kubernetes cluster.

Vendor-neutral:

Gateway API is designed to be vendor-neutral, meaning it doesn't favor any specific networking solution. This ensures that it can be used consistently across different environments and networking solutions, enhancing portability and reducing vendor lock-in. This neutrality also fosters a more collaborative ecosystem, as improvements and extensions can benefit all users, regardless of their specific networking implementation.

Role-oriented

In the context of the Kubernetes Gateway API, being "role-oriented" means that the API is designed with distinct roles for different types of users. This approach allows for a more fine-grained control over resources, improving security, and making the management of permissions more straightforward.

For example, the roles could be divided between cluster operators and application developers. Cluster operators control infrastructure-related aspects of gateways and routes, such as selecting the type of load balancer used, setting up SSL/TLS certificates, and managing access logs. On the other hand, application developers manage application-level configurations, such as specifying the routes for their applications, setting up path-based routing, and defining request and response headers.

This role-oriented design allows each user to focus on their area of expertise, without needing to worry about the details that are outside of their purview. It also ensures that only authorized users can make changes to specific aspects of the configuration, enhancing the overall security of the system.

Why the Managed Gateway is Needed

Let's move on to the next section and discuss why a Managed Gateway is needed. As you can guess from its name, it simplifies our work.

why multi-tenant Gateways are needed

The Managed Gateway is essential for several reasons:

Simplified Management: Managed Gateways handle the underlying infrastructure, freeing up development teams to focus on application logic. This simplifies the management of the gateway, as the complexities of configuration, maintenance, and upgrades are handled by the provider.
Scalability: Managed Gateways offer automatic scaling capabilities. They can scale up to handle high traffic loads and scale back down during quieter periods, ensuring optimal resource usage.
Reliability: Managed Gateways provide high availability and fault tolerance. They are designed to maintain service continuity, even in the face of network disruptions or hardware failures.
Security: Managed Gateways often come with built-in security features such as SSL encryption, and identity-based access control. This ensures that your applications are secure and compliant with industry standards.
Isolate and protect tenant traffic, reducing the blast radius.

General solution

To handle this scenario, they will deploy multiple ingress controllers. Here are two common solutions:

Differentiation through IngressClass: IngressClass is a Kubernetes resource that allows users to specify the type of Ingress controller they want to use in their applications. By defining different IngressClasses, users can associate their Ingress resources with specific Ingress controllers. This is beneficial as it allows for a clear separation and management of different application traffic within the cluster, each handled by its own Ingress controller.
Differentiation through namespaces: Another method to differentiate between multiple Ingress controllers is by using namespaces. Kubernetes namespaces provide a scope for names and allow users to divide cluster resources between multiple users or teams. By deploying different Ingress controllers in different namespaces, users can ensure that each controller only manages the traffic for the applications within its own namespace. This approach enhances the security and organization of the cluster, as it provides a clear separation of concerns between different applications and their respective Ingress controllers.

Pain point

While deploying multiple Ingress controllers in the same cluster may seem like a viable solution, it introduces several pain points:

Increased Complexity: Managing multiple Ingress controllers increases the complexity of your Kubernetes cluster. Each controller has its own configuration and behavior, leading to potential inconsistencies and conflicts. This can make it hard to predict how the controllers will interact and complicate debugging efforts when issues arise.
Resource Wastage: Each Ingress controller requires its own resources to run. Deploying multiple controllers may lead to resource wastage, especially if some controllers are not fully utilized. This inefficiency can lead to increased costs and reduced performance for your cluster.
Increased Maintenance: With multiple Ingress controllers, you need to maintain each one separately. This includes monitoring, updating, troubleshooting, and securing each controller. This increased maintenance can take significant time and effort.
Lack of Centralized Control: Having multiple Ingress controllers can make it difficult to achieve centralized control and visibility over your network traffic. Without a single point of control, it can be challenging to manage traffic routing consistently and apply uniform security policies across all controllers.
Potential Security Risks: Each Ingress controller has its own security features and configurations. If not properly managed, having multiple controllers can introduce security risks, as each controller could potentially be a separate point of vulnerability in your cluster.

Kong Gateway Operator

Today, let me share an innovative open-source project created by Kong Inc., known as the Kong Gateway Operator. This project is not just another tool in the tech space; it's a game-changer designed to address some of the most pressing challenges we face in managing API gateways.

In our journey towards digital transformation, we often encounter bottlenecks that hinder our progress. The Kong Gateway Operator is here to eliminate those bottlenecks by offering:

Full Lifecycle Management: Managing the lifecycle of Kong Gateway has never been easier. From deployment to retirement, the Kong Gateway Operator ensures a smooth journey. Moreover, it supports a blue/green strategy for upgrading Kong Gateway, making transitions seamless and minimizing downtime.
Elastic Scaling: Cost-efficiency is paramount in today's tech landscape. The Kong Gateway Operator leverages Horizontal Pod Autoscaling (HPA) and latency measurements to dynamically scale your resources. This not only ensures optimal performance but also significantly reduces operational costs.
Automatic Certificate Rotation: Security is a top priority, and managing certificates can be a cumbersome task. Thanks to the integration with cert-manager, the Kong Gateway Operator automates certificate rotation, ensuring your applications are always secure without the manual hassle.
AI Gateway Support: In an era where AI and LLM (Large Language Models) applications are becoming increasingly prevalent, the Kong Gateway Operator sets the stage by prioritizing AI gateway support. This feature is designed to streamline the development and deployment of AI-driven applications, making it easier for developers to integrate cutting-edge technologies into their solutions.

How to Deploy a Managed Gateway using KGO in Civo

Create Kubernetes cluster

The cluster running on Civo is k3s, but it is also a distribution that has passed Kubernetes conformance certification.

To create a cluster through Civo, you can operate it in the dashboard of Civo or install Civo's CLI. It is very convenient to create through CLI. Here I have installed and configured Civo's CLI.

➜  ~ civo k8s create --merge  --save  --wait
Merged with main kubernetes config: /home/tao/.kube/config

Access your cluster with:
kubectl config use-context red-nightingale-6cf01085
kubectl get node                     
The cluster red-nightingale-6cf01085 (9752f456-f316-4bdf-95fe-82c9b08db61b) has been created in 1 min 9 sec

Deploying clusters on Civo is very fast, which is one of the reasons why I like Civo the most.

Install KGO

KGO can be installed through Helm, and the CRDs of Kubernetes Gateway API are already included in KGO's Helm chart, so there is no need to perform a separate installation step.

Add kong Helm repo.

(⎈|red-nightingale-6cf01085:default)➜  ~ helm repo add kong <https://charts.konghq.com>
"kong" has been added to your repositories
(⎈|red-nightingale-6cf01085:default)➜  ~ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kong" chart repository
Update Complete. ⎈Happy Helming!⎈

Install KGO using Helm.

(⎈|red-nightingale-6cf01085:default)➜  ~ helm upgrade --install kgo kong/gateway-operator -n kong-system --create-namespace --set image.tag=1.3

Release "kgo" does not exist. Installing it now.
NAME: kgo
LAST DEPLOYED: Sat Jul 20 01:40:28 2024
NAMESPACE: kong-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
kgo-gateway-operator has been installed. Check its status by running:

  kubectl --namespace kong-system  get pods

For more details, please refer to the following documents:

* <https://docs.konghq.com/gateway-operator/latest/get-started/kic/create-gateway/>
* <https://docs.konghq.com/gateway-operator/latest/get-started/konnect/deploy-data-plane/>

Create Gateway

To use the Gateway API resources to configure your routes, you need to create a GatewayClass instance and create a Gateway resource that listens on the ports that you need.

(⎈|red-nightingale-6cf01085:default)➜  ~ echo '
kind: GatewayConfiguration
apiVersion: gateway-operator.konghq.com/v1beta1
metadata:
 name: kong
 namespace: default
spec:
 dataPlaneOptions:
   deployment:
     podTemplateSpec:
       spec:
         containers:
         - name: proxy
           image: kong:3.7.1
           readinessProbe:
             initialDelaySeconds: 1
             periodSeconds: 1
 controlPlaneOptions:
   deployment:
     podTemplateSpec:
       spec:
         containers:
         - name: controller
           image: kong/kubernetes-ingress-controller:3.2.0
           env:
           - name: CONTROLLER_LOG_LEVEL
             value: debug
---
kind: GatewayClass
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
 name: kong
spec:
 controllerName: konghq.com/gateway-operator
 parametersRef:
   group: gateway-operator.konghq.com
   kind: GatewayConfiguration
   name: kong
   namespace: default
---
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
 name: kong
 namespace: default
spec:
 gatewayClassName: kong
 listeners:
 - name: http
   protocol: HTTP
   port: 80

' | kubectl apply -f -

gatewayconfiguration.gateway-operator.konghq.com/kong created
gatewayclass.gateway.networking.k8s.io/kong created
gateway.gateway.networking.k8s.io/kong created

You can verify whether the Gateway has been successfully created by using the following command.

(⎈|red-nightingale-6cf01085:default)➜  ~ kubectl get Gateway -A
NAMESPACE   NAME   CLASS   ADDRESS         PROGRAMMED   AGE
default     kong   kong    74.220.29.143   True         3m35s

(⎈|red-nightingale-6cf01085:default)➜  ~ export PROXY_IP=$(kubectl get gateway kong -n default -o jsonpath='{.status.addresses[0].value}')
(⎈|red-nightingale-6cf01085:default)➜  ~ curl $PROXY_IP                                                                                   
{
  "message":"no Route matched with those values",
  "request_id":"382d5a15ac243b3183b5b239a5e3ae77"
}

At the same time, you can also see that KGO has created deployments for CP and DP in the default namespace.

(⎈|red-nightingale-6cf01085:default)➜  ~ kubectl get deploy          
NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
dataplane-kong-tkbct-snkrk      1/1     1            1           9m5s
controlplane-kong-lv62b-tf6xl   1/1     1            1           9m5s

Create another Gateway

To verify the ability of KGO, we have created a new namespace and created Gateway resources.

(⎈|red-nightingale-6cf01085:default)➜  ~ kubectl create ns moe        
namespace/moe created

(⎈|red-nightingale-6cf01085:default)➜  ~ echo '
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
 name: moe
 namespace: moe
spec:
 gatewayClassName: kong
 listeners:
 - name: http
   protocol: HTTP
   port: 80

' | kubectl apply -f -

We can check whether Gateway has been deployed correctly by viewing the status of Gateway resources, or we can verify it by requesting the IP of Gateway using the method in the previous section.

(⎈|red-nightingale-6cf01085:default)➜  ~ kubectl get Gateway -A
NAMESPACE   NAME   CLASS   ADDRESS         PROGRAMMED   AGE
default     kong   kong    74.220.29.143   True         14m
moe         moe    kong    74.220.28.102   True         8m6s

Of course, a more intuitive way would be to check the status of kocp and kodp.

(⎈|red-nightingale-6cf01085:default)➜  ~ kubectl get kodp,kocp -A
NAMESPACE   NAME                                               READY
default     dataplane.gateway-operator.konghq.com/kong-tkbct   True
moe         dataplane.gateway-operator.konghq.com/moe-bfwnl    True

NAMESPACE   NAME                                                  READY   PROVISIONED
default     controlplane.gateway-operator.konghq.com/kong-lv62b   True    True
moe         controlplane.gateway-operator.konghq.com/moe-9dgjc    True    True

After the Gateway is ready, you can refer to the KGO official documentation to use Kubernetes Gateway API to create routes.

Summary

In this article, I introduced why we need to use a multi-tenant Gateway in Kubernetes clusters, and also demonstrated how to deploy the Gateway through KGO in Civo's Kubernetes cluster.

KGO provides a wide range of capabilities, please feel free to explore them! https://konghq.com/products/kong-gateway-operator

The Ultimate Guide to Kubernetes Networking and Multi-Tenant Gateways

Table of contents