Docker Architecture

Docker Components

• Docker Architecture Docker is a platform for building, running, and shipping applications in lightweight, portable containers. It abstracts the operating system and allows applications to run consistently across different environments. Here's a detailed look at its architecture:

1. Docker Client

The Docker client is the interface through which users interact with Docker. It can be run on the same machine as Docker or remotely. The client sends commands to the Docker daemon and can be accessed via:

• CLI (Command Line Interface): Users issue commands like docker build, docker run, etc.

• Docker API: Programmatic interface for controlling Docker.

2. Docker Daemon (dockerd)

The Docker Daemon is the core part of Docker. It is responsible for managing Docker containers, images, networks, and volumes. The daemon listens for Docker API requests and is usually responsible for:

• Creating Containers: Running a container from an image.

• Building Images: Building a new image from a Dockerfile.

• Managing Networks and Volumes: Connecting containers to networks and managing persistent data.

The Docker Daemon communicates with the Docker Client, Docker Hub (or other registries), and other components.

3. Docker Images

Docker images are read-only templates used to create Docker containers. An image includes everything needed to run an application—code, runtime, libraries, dependencies, and configuration files. Images are typically built using a Dockerfile.

• Layered Architecture: Docker images are made up of multiple layers (e.g., base OS, dependencies, application code). Each layer is cached to improve performance and efficiency.

4. Docker Containers

A Docker container is a running instance of a Docker image. Containers are isolated from each other, providing a lightweight virtualized environment for applications.

• Namespaces: Ensure process isolation.

• Control Groups (cgroups): Manage resource allocation such as CPU and memory usage.

• Union File Systems: Allow layers in images to be merged together.

5. Docker Hub/Registry

Docker Hub is a cloud-based registry for storing and sharing Docker images. However, you can also use a private registry or other services like GitLab’s Container Registry.

• Docker Push: Push your custom images to a registry.

• Docker Pull: Retrieve images from a registry to use in containers.

6. Docker Networks

Docker provides networking features that allow containers to communicate with each other. There are several types of networks:

• Bridge Network: Default network type where containers are isolated but can communicate with each other.

• Host Network: The container shares the host's network stack.

• Overlay Network: Used to connect containers across multiple Docker hosts.

7. Docker Volumes

Volumes are used to persist data beyond the lifecycle of a container. They are stored outside the container and can be shared across containers.

Docker Lifecycle

• 🧑‍💻 Developer writes a Dockerfile

  • Contains instructions (base image, dependencies, commands)

• 🏗️ Build image with docker build

  • Docker CLI sends build request → Docker Daemon reads Dockerfile → creates an image layer by layer

• 📦 Image gets stored locally

  • Can be pushed to a remote registry (e.g., Docker Hub)

• 🚀 Container is started using docker run

  • Docker Daemon creates a container from the image → allocates resources (namespaces, cgroups)

• 🔄 Docker Engine manages container runtime

  • Handles isolation, networking, volume mounting, process management

• 📡 Optional networking and volumes

  • Docker sets up virtual networks, attaches storage as needed

• 🗂️ Logs, stats, and monitoring available

  • Docker provides tools to monitor and manage the container

• 🧹 Container can be stopped or removed

  • Temporary containers can be cleaned using docker rm

Docker Flow Diagram

---

Kubernetes Architecture

Kubernetes Components

• Kubernetes Architecture Kubernetes (K8s) is a container orchestration platform that automates the deployment, scaling, and management of containerized applications. It runs on top of Docker (or other container runtimes) and adds additional functionality for distributed systems. Below is the detailed architecture of Kubernetes:

1. Kubernetes Master Node

The master node is the control plane that manages the Kubernetes cluster. It is responsible for managing and maintaining the desired state of the cluster. The main components of the master node include:

• API Server (kube-apiserver): This is the front-end for the Kubernetes control plane. All REST API calls go through the API server. It validates and processes the requests and maintains the desired state of the system in etcd.

• Controller Manager (kube-controller-manager): Manages controllers that regulate the state of the cluster. Each controller is responsible for a specific aspect of the cluster, such as ensuring that the number of replicas of a pod matches the desired number.

• Scheduler (kube-scheduler): Decides where to place newly created pods (based on available resources, constraints, etc.). It assigns pods to nodes in the cluster.

• etcd: A consistent and highly-available key-value store that Kubernetes uses to store all of its cluster data (e.g., configuration data, state of the cluster, etc.).

2. Kubernetes Worker Nodes

The worker nodes (also called minions) are responsible for running the containers. Each worker node has the following components:

• Kubelet: An agent that runs on each worker node and ensures the containers are running in the desired state. It manages the lifecycle of the containers and reports to the master node.

• Kube Proxy: Maintains network rules for pod communication across the cluster. It also facilitates load balancing and routing requests to appropriate pods.

• Container Runtime: This is the software responsible for running the containers. It could be Docker, containerd, or any other container runtime.

3. Kubernetes Pods

A pod is the smallest deployable unit in Kubernetes. It can contain one or more containers, which are managed together. Pods are scheduled on nodes and share resources such as:

• Network: All containers in the pod share the same network IP.

• Storage: Containers in the same pod share the same volumes.

Pods are ephemeral in nature, meaning they can be destroyed and recreated as needed.

4. Kubernetes Services

A service is an abstraction that defines a set of pods and a way to access them, usually via a stable IP address or DNS name. Services allow for load balancing and service discovery. There are different types of services in Kubernetes:

• ClusterIP: Exposes the service on a cluster-internal IP.

• NodePort: Exposes the service on each node’s IP at a static port.

• LoadBalancer: Exposes the service externally using a load balancer.

• ExternalName: Maps the service to an external DNS name.

5. Kubernetes Deployments

A deployment is a higher-level abstraction that manages replicas of pods. It allows for:

• Scaling: Increase or decrease the number of pods.

• Rolling Updates: Update the application without downtime.

• Self-healing: Automatically restarts or reschedules pods in case of failures.

6. Kubernetes ReplicaSets

A ReplicaSet ensures that a specified number of pod replicas are running at all times. If a pod crashes or is deleted, the ReplicaSet will create a new pod to replace it. Typically, ReplicaSets are managed by Deployments.

7. Kubernetes Namespaces

Namespaces are a way to divide cluster resources between multiple users or applications. They provide isolation for different projects or teams working in the same cluster.

8. Kubernetes ConfigMaps and Secrets

• ConfigMaps: Store configuration data that can be used by applications.

• Secrets: Store sensitive information like passwords or API keys.

9. Kubernetes Ingress

Ingress allows external HTTP/S traffic to reach services within the cluster. It provides routing, SSL termination, and more advanced features like load balancing.

Kubernetes Lifecycle

1. User creates deployment (kubectl apply)

2. kube-apiserver stores it in etcd

3. Scheduler assigns pod to a node

4. kubelet pulls image, starts containers

5. Probes ensure pod is healthy

6. Pod serves traffic via Services

7. Auto-scaling, updates, self-healing as needed

8. Pod is terminated or replaced (update/failure)

9. Cleanup via finalizers & garbage collector

Kubernetes Flow Diagram

Key Points-

1. One master node can have a multiple worker node.

2. Control Plane = Master Node.

3. Kubernetes Minions = Worker Node.

4. Once k8s installed, all 4 components will get installed i.e., (kube-apiserver, kube-schedular, etcd and kube-controller-manager).

5. kube-apiserver is responsible to connect with all other 3 components and also responsible to connect with worker node’s kubelet and kube-proxy.

6. kube-scheduler > it will schedule the pods.

7. PODS: It is the smallest deployable unit.

8. 1 worker node can have multiple pods.

9. 1 pod can contain multiple containers and those containers will connect with each other on the same IP address and they will interact on the same local host.

10. Kubelet: It is just like an agent, which holds the interaction between the master node and worker node.

11. etcd: It stores all the sensitive information like- admin password, pods/cluster information.

12. kube-controller-manager: It helps to maintain the desired state of the cluster. Let’s say, I want to run 4 pods at a time but somehow 2 pods got down, so this controller manager will again create the 2 pods to maintain the desired state. There are multiple types of controller manager like: Namespace controller manager, Volume controller manager, Replication controller manager.

13. kube-proxy: It is a DNS manager of k8s. It assign’s the IP address, how to interact services to the pods, if request is coming then route it to the desired pod.

Source: https://youtu.be/2s3o5xvvqfE?si=JRJmCqTqW3O5S4AH

---

Intro: HELM Chart

What is Helm?

• Helm helps you manage Kubernetes applications.

• Helm is a package manager for Kubernetes. It helps you define, install, and upgrade even the most complex Kubernetes applications. Helm uses a packaging format called charts.

What is a Helm Chart?

• Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.

• A Helm Chart is a collection of YAML files that describe a related set of Kubernetes resources. It acts like a blueprint for deploying applications.

  • Think of it like:

    Helm Chart = Dockerfile for Kubernetes deployments

• Charts are easy to create, version, share, and publish.

• Bundle of YAML Files.

• Create your own Helm Charts with Helm.

• Push them to Helm Repository.

• Download and use existing ones.

Use Case of Helm Chart

Reusable File Use Case:

Let’s say I have a large-scale application running on Kubernetes, consisting of multiple components and services. Now, I need to introduce a new service that also includes several interdependent components. Traditionally, I would have to manually create multiple Kubernetes YAML files (like Deployments, Services, ConfigMaps, etc.) and apply them one by one using kubectl.

Managing such services becomes tedious and error-prone, especially when making configuration changes — as I’d need to update each file individually. Similarly, if I ever need to delete the entire service, I’d have to manually delete all related resources, which is time-consuming and prone to oversight.

This is exactly where Helm Charts come into play. Helm allows us to define all the necessary Kubernetes resources in a single, reusable chart. Configuration values are centralized in a values.yaml file, making it easy to update parameters across multiple components. Deploying the service becomes as simple as running a single Helm command, and cleaning it up later is just as easy — we can remove all associated resources by simply uninstalling the Helm release. Helm brings versioning, templating, and lifecycle management to Kubernetes deployments, drastically simplifying the process of managing complex applications.

Blueprint File Use Case:

Another common use case involves managing multiple microservices where each service has its own Kubernetes YAML configuration. Often, these YAML files are nearly identical, with only a few lines differing — such as image names, ports, or environment-specific settings.

Using Helm, we can streamline this process by creating a common blueprint in the form of templates. Instead of duplicating entire YAML files for each microservice, we define reusable templates where the variable parts are replaced with placeholders. These placeholders are dynamically populated using values from the values.yaml file during deployment. This significantly reduces redundancy, improves maintainability, and allows consistent updates across all services with minimal effort.

Template Engine Use Case:

Helm Chart Structure:

---