Using Istio Gateway to Route Traffic to Microservices on Amazon EKS

I think it's a safe guess that most workloads running on Kubernetes are based on microservice architectures. This is largely due to the fact that microservices translate really well to containers. However, one of the glaring challenges of deploying microservices to Kubernetes is figuring out network communication from the outside world into your clusters, specifically to the services inside of it, as well as network communication between the services themselves.

In this video, I'll discuss and demonstrate how you can use Istio to manage the traffic routing to your microservices in an Amazon EKS cluster. In Kubernetes, we have services to expose our applications, which are stable network abstraction layers that sit in front of our pods and keep track of a table of IPs for the running pods they're attached to. To expose our apps to the outside world, we typically use LoadBalancer services and NodePort services. While these can work well in some cases, we need to be aware of their shortcomings.

For example, creating an external load balancer for every service is not cost-effective, whether you have a small set or a large set of microservices. Opening a node port for every service is also not secure, as it increases the attack surface area with each open port. Moreover, these services only listen for traffic on Layer 4 of the network OSI model (TCP or UDP), but there are scenarios where you want to handle HTTP traffic.

What you want is a single point of entry into your cluster where you can implement security measures and controls and smart routing to the relevant microservices. This setup allows your pods to be fronted by ClusterIP services that will proxy the traffic through to the pods. This is where Istio comes in.

Istio is an open-source implementation of a service mesh, which is a distributed middleware solution that sits at the infrastructure level. Its job is to handle all the network communication responsibilities on behalf of your applications. We can use it to manage things like service discovery, load balancing, health checks, throttling, tracing of network requests, retries, and more, without needing to build these capabilities into the application itself.

Istio's architecture consists of a data plane, a control plane, and ingress and egress gateways. The data plane includes service proxies that live alongside the applications they manage network responsibilities for. The control plane has various components that expose an API for configuring the behavior of the data plane components. The ingress and egress gateways manage traffic flowing into and out of the mesh.

To simplify the concept, think of your Kubernetes cluster as a company. Each microservice is a business person, and you need a better way to secure and manage communication within the company and with the outside world. The service mesh acts as a team of personal assistants (the data plane) managing communication for each business person. The control plane is the system that coordinates the assistants. The ingress and egress gateways are like guards managing all incoming and outgoing communication.

Now, let's look at my example. You'll see that the Istio ingress gateway can expose an external load balancer, routing traffic through the gateway into your cluster. In the AWS context, this is a Classic Load Balancer with a virtual IP that proxies traffic to different instances, protecting you from a single point of failure. I have a specific hostname mapped to this load balancer's virtual IP in Amazon Route 53. Client requests to the hostname (e.g., lukemilo.com) go through Route 53 DNS servers to the load balancer and eventually to the Istio ingress gateway, the entry point for traffic into the cluster and service mesh.

The gateway resource in Istio configures the ingress gateway, specifying the ports to open and the virtual hosts to receive traffic. Virtual services, on the other hand, configure traffic routing rules, specifying how sidecar service proxies interact with services in the mesh. For instance, I have virtual services for three microservices in my example: GraphQL, orders, and products. These virtual services specify rules for traffic routing, retry configurations, and destination services.

Switching to my VS Code editor, I'll show the gateway and virtual service configurations. The gateway resource opens port 80 for HTTP traffic to the hostname lukemilo.com. The virtual services configure the sidecar proxies for traffic routing to the respective microservices. For example, the GraphQL virtual service routes traffic to the GraphQL service, with retry configurations for resilience.

Now, let's switch to the browser. Here, you can see my EKS cluster (Alpha), running Kubernetes version 1.21, with a single node group of four T3 medium nodes. In Route 53, my A record maps lukemilo.com to the Istio ingress gateway's external load balancer. The GraphQL endpoint GUI shows data from the orders and products microservices, aggregated by the GraphQL service. The playground endpoint provides a UI for interacting with the GraphQL service.

Lastly, using k9s, you can see the running pods in the e-commerce namespace for GraphQL, orders, and products, as well as the Istio ingress and egress gateways. The virtual services for these microservices manage how the sidecar proxies route traffic to the destination services.

There you have it: a comprehensive look at using Istio to manage traffic routing for microservices in a Kubernetes cluster.