Kubernetes 101: Part 5

Cluster Networking

Within a cluster, we have master node, worker nodes . Each node carries pods.

The whole cluster is connected to a network which assigns IP address to the nodes.

Typically this is the master node and worked node looks like. Master node has kube-api, kubelet (can be there), kube-scheduler, kube-controller-manager. Other external nodes, agents etc contacts kube-api to get work done.

The ports for kube-api is 6443, for kubelet it’s 10250, for kube-scheduler it’s 10259, for kube-controller-manager it’s 10257.

The worker node expose ports for external services. The services can have a port between 30000-32767

The ETCD server on master node is at 2379 port

If we have multiple master nodes, we need to create one additional ETCD Client at port 2380 to communicate both masters.

Pod Networking

There are pods within nodes and every pod should contact each other.

By default kubernetes does not solve this issue and expects external solution to solve this issue.

But the solution must follow these rules

There are solutions like these to use

But how do they do it? Let’s see

Firstly each node gets assigned to some IP address

Then when the pods are created, kubernetes create network namespace for them.

To enable communication between them, we add these namespace to network through bridge.

We create bridge network on each node

Let’s have each bridge network on its own subnet. Assuming they are 10.244.1, 10.244.2, 10.244.3

Next is to set up the IP address

Now the basic tasks are done. The remaining tasks are to be performed by each container and every time a container is created. A script will do this for us.

To attach a container to the network, use

ip link add …

Then one end to the container and another end to the bridge . Use

ip link set ….

Then we assign IP address and

ip -n <namespace> addr add….

add a route to the default gateway.

ip -n <namespace> route add ….

But what IP address to add? For now, let’s use 10.244.1.2

Then we bring the interface

ip -n <namespace> link set ….

So, the blue container (left most) is connected to the bridge

We will repeat this script for the another container.

Now both blue and orange containers can communicate and now they have their own IP. We can use this script for other nodes and make sure other containers can talk within themselves.

The containers within the node can communicate but can they do it with different containers outside of their node? For example, here blue pod wants to connect to purple pod

It does not work!! To solve this issue, we can create a route within our desired pod (purple'; 10.244.2.2) and the Node 2 IP

Assuming blue pod is added with Node1 and using LAN, nodes can communicate and therefore, the pods can too.

Now you can see them communicate.

Then we can create route for all of the container (hosts) with their Node IP.

or, we could use a route table to manage these

The problem will be solve using route table as well.

Now, remember we used a script to work with containers and network?

But that script runs on CNI which has it’s format.

ADD is to add containers to the network and DEL to delete container interface from the network.

So, when a new container is created, the kubelet looks at the CNI configuration passed as a command line argument when it was run and identifies our script name. (/etc/cni/net.d/………). It finds the script in the cni’s bin folder (/opt/cni/bin/n…)

Then it executes it with add command using name and namespaceID of the container.

CNI in kubernetes

CNI basically defines the responsibilities of container runtime. CNI defines the responsibilites of container runtime. Here, kubernetes is used as container runtime which will be responsible to container network namespace, indentify and attach those namespaces to the right network by called the right network plugin.

So, where is this CNI plugin created?

Basically it’s configured in the kubelet of each node in the cluster.

One of the tools which works as a CNI solution is , Weaveworks

Let’s understand how this works in our system. So, our blue pod in Node1 wants to send a ping to purple pod. Using the route table, the connection is defined.So, the ping has been sent.

It was okay because we dealt with just 2 containers/pods.

But this is not feasible for big clusters. We have lots of containers in nodes and specially cluster. So, we can’t keep so many container information only in routing table. It will be a mess!

Let’s use another technique

Assume that we have different offices at different places. And surely each office has some department. Now assume these departments want to send packet to each other.

Someone in office-1 wants to send a packet to office-3

The packet then passes via the connection to the goal

We just had 3 nodes and it was easy to deal. But what if we have too much nodes? Then we should use a 3rd party solution which can manage these tasks for us

For example, while using tools like weaveworks , it keeps agents in all of the nodes

The agents are well connected with their internal network. So, now if any office/pod/container wants to send a packet, the agents know where to send the packet and how easily it can be done.

Here, office-10 wants to send to office-3

The agent creates a package and keeps the packet within that. The package has destination and from set.

Then weaveworks carries the package

The packet is passed to the agent on the node. Then the agent unpacks it and give packet to the office

So, this is it. A 3rd party tool like installs it’s agents once it’s used

You can also see how they (weaveworks agents) are connected within themselves via internal network.

But how to deploy that on a cluster?

We can use this command to deploy this as a service/ daemon set.

Then one copy of pod (agent) is created in every node.

DNS

Assume that we have 2 pods and 1 service which has their own IP (Note: They might be in different nodes)

To connect the web server with test one, we use web-service. Whenever a service is created, Kubernetes DNS service creates a record for the service. It maps out the service name to the IP address.

So, using any pod/container , we can reachout to the service by it’s name (as DNS server has it’s name and IP in record)

What if they are part of two different namespace? Then you need to specify with DNS name.namespace name

Here from the test pod, we have to use web-service.apps to get access to the web-service within the apps namespace

But all the services are grouped together within a namespace. For services, it’s svc subdomain group

So, we need to use service-name.namespace.subdomain-name

Finally, all the pods and services are grouped together within root domain for the cluster.

So, now to get access to the web-service we need to specify web-service-name.namespace-name.service-group.pod and service group. That’s it for services.

How to connect to pods using curl?

Although that’s not done in general, kubernetes uses the IP of the pod to create a name in the DNS. for example, the IP of web is 10.244.2.5 which turns to 10-244-2-5

And pods are part of pod group and also part of the cluster.local group which has all pods and services.

This is how test pod can contact to the web pod kept in apps namespace

Also, same goes for the test pod if someone wants to contact to this pod

It’s part of the default namespace and part of pod group and surely part of the cluster.local group within the default namespace.

Ingress

Assume that you have an online store

So, you can create this application pod (“wear”) which will have all of these images

Then your application needs a database and thus you created a pod MySQL. To connect to the database pod with our online store content pod (“wear”), there is a service “mysql-service” is created of type ClusterIP.

Again to make the application pod available from outside, you created another service (“wear-service”) and connected that to port 38080 which is a NodePort and ranges between 30000-32767

When the traffic increases, we increase replicas

Now, we don’t want users to use <node-ip>:3808 instead we can point to IP table and users can then use the my-online-store.com:38080

We also don’t want users to remember the Nodeport as it’s a big number. To solve this, we will follow 2 ways.

Way 1: we can add a proxy-server which will connect to the NodePort. Then we can route the traffic to proxy-server

As port 80 is a default port, so users don’t need to mention port any more. If they search with my-online-store.com, they will be sent to proxy-server.

Way 2:

If we use cloud platforms like GCP, a load balancer can be added here. It will exactly work as the proxy-server and users can reach the website using my-online-store.com

Assume that you want to introduce new service like live watching

How will the architecture change now?

Based on the way 2, we can introduce another load balancer with the pods of video stream

But we have 2 services now. We need to send traffic to wear or video pods depending on user’s need. To solve this issue, another load balancer can be added which will redirect traffic based on the need

We then add this load balancer in the DNS. Also, we need to enable SSL certificate to make our my-online-store.com much secured and can be used using https

To do all of these things, we can use ingress

We need to deploy ingress solution which is called as ingress controller

and rules which is called as ingress resources.

Among these ingress controllers, GCP load balancer and NGINX is maintained by Kubernetes. Let’s use NGINX by deploying this for ingress-controller.

But we are not done here. NGINX has set of configuration options such as path to store the logs, keep alive threshold, SSL Settings, session timeout etc.\

We have ton create a configmap object pass the nginx controller image to decouple these configuration.

Also, we need to pass 2 environment variables which carries the pod’s name and namespace it’s deployed to.

The nginx pod need these 2 data to read content from the pod. Then the port used by this ingress controller

Then we will create a service to expose this nginx controller to the world

The ingress controller have additional intelligence built into them to monitor the kubernetes cluster for ingress resources and configure underlying nginx server when something has changed. To do this, we we need service account with right permissions.

Here a service account is created with correct roles and role bindings

So, overall this is how we created the ingress controller

Let’s create ingress resource (rules on ingress controller) now

There can be different types of rules we may set

Our goal was to redirect traffic based on the path (/wear or /watch)

To do that, we can have 1 rule as we have 1 domain and 2 paths (for wear and watch)

For the wear and watch pod, we can set this rule to redirect

Here in the image, you can see service infront of pods. So, the yaml file also redirects traffic based on paths (/wear, /watch) to the service (wear-service, watch-service)

Once we run it and check description

We can see 2 URLs in the Backends.

So, this is our desired output once user goes for wear or watch path.

But if the user asks for a different path which we don’t have?

We can then set a 404 page and redirect for such request.

We also have another ways to solve the whole scenarios.If we had dedicated dns for wear and watch pages, we can now use them in hosts and just redirect them directly to the services responsible for the pod. (Note: This image does not show the service , here you can just see wear and watch pod. But don’t worry, the services are there to connect to the pods.)