Kubernetes 101: Part 10

Assume you team wants to release a new feature for an app but it takes a lot of time to fix bugs, test them etc.

To solve this issue, we can use Continuous Integration (CI) and Continuous Deployment (CD)

CI ensures that every code is built and tested and containerized while the code is getting developed. So, issues are addressed early and solved too.

With CD, the new deployments are pushed to the production Kubernetes cluster.Which makes faster delivery.

So, here comes GitOps which ensures that you can manage your infrastructure and deployment process as code. This ensures that the changes to my infrastructure are tracked and version controlled. We can easily rollback to changes if something goes wrong.

What is GitOps?

Assume that you have a kubernetes cluster with deployment, services, configmaps etc

Now your team mates made changes to config map and deployment directly without notifying you.

That made the whole thing crash

But if you and your use GitOps, this issue is no more!

With GitOps, team members make changes to kubernetes object by commiting changes to a git repository. When a change is committed, a GitOps operator or Flux or Argo CD automatically pulls the changes from the git repository and deploys the infrastructure changes to the live environment.

So, in case of the config map update by your teammate, he would now create a PR and once approved only then it would get deployed.

GitOps tools

GitOps tools are Flux, Argo, JenkinX

Flux

Flux watches at a Git repo for changes using pull based model and periodically syncs any changes with the Kubernetes environment. The issue with flux is that, it can just watch one git repo and deploy to only one Kubernetes cluster, namespaces. Flux focuses on CD (Continuous Deployment)

Argo

Argo is similar as Flux and a declarative tools designed for Kubernetes. It also works as a pull based model and focuses on Continuous Deployment. A single Argo Installation can monitor multiple git repos and deploy to multiple kubernetes namespace which makes it more flexible.

JenkinX

It’s focuses on CI/CD. It’s simple to deploy but more complex than Argo and Flux.

GitOps Principle

GitOps should have a working group/working body.

A working group works under CNCF app delivery SIG which aims to define a vendor neutral meaning of GitOps. The goal is to create a common language and understanding of GitOps that can be used by everyone in the community.

The GitOps working group started Open GitOps Project to define and standardize best practices for GitOps. It came up with 4 rules:

1) The entire system has to be described declarative. Four example, instead of manually installing application, we use yaml file to specify the desired state of the application.

2) Version and immutable

Once we have defined desired state of our application in the YAML file , we commited to git to version control it. This allow us to track changes to the file over time and to revert to previous version if needed.

3) Pulled automatically approved changes.

To deploye our application we use Flux or ArgoCD to automate the process. When we push the yaml file to git, the GitOps tools (Flux, argocd) detects the changes with a pull based model and automatically applies it to the kubernetes cluster.

4) Continuously reconcile software agents to ensure correctness and alert on divergence. Once the application is up and running , we want our application to remain in the desired state defined in the yaml file. To monitor that, we use GitOps tools to monitor the kubernetes cluster and compare its actual state to the desired state in the yaml file in git. If anything wrong happens, it takes actions to make it correct

Push vs Pull based Deployments

There are 2 approaches. 1) Pull based 2) Push based

1. Push based:

Here an external system such as a continuous Delivery(CD) Pipeline triggers the deployments to the kubernetes cluster after a successful commit to a git repository or a previous CI pipeline

If we choose this approach, we need to explore the cluster credentials outside the cluster and store them in the CI system so that CD Pipeline (as CD pipeline need access) can have direct access to the cluster and push the changes to the production environment.

Make sure to take security measurements when exposing credentials

2. Pull based approach:

Here all changes are made inside the kubernetes cluster. This is done using an operator which scans associated git repositories and docker registries for changes.

If a change is detected, the operator updates the cluster’s data accordingly.

One of the benefit of this approach is, it’s security as no credential is getting exposed.

CI/CD with GitOps

In GitOps, we maintain 2 repository.

In application code repository, we keep code, data, images, kubernetes manifest.

Whereas in the kubernetes manifests repository, we keep yaml file that describes desired state of the cluster. It inclused deployment, service, ingress, configmap secret and other kubernetes resources.

To apply GitOps, we use ArgoCD which monitors the kubernetes manifest repository for changes and reconcile the desired state with the actual state of the cluster. It deploys updates and release resources as necessary to achieve the desired state.

Assume a developer made new changes to the application code repository.

To deploy this changes, we use CI pipeline that consists of unit tests, building artifacts, and creating a docker image.

The CI pipeline then pushes the newly build image to the container registry.

Once the image is available, in the container registry, we need to update the kubernetes manifest repo wiht a new image version. This can be done by modifying the yaml files that reference the image and committing the changes to the repository.

Then we raise a PR to kubernetes manifest repo that includes these changes we made.

The PR is reviewed and approved by manager or architect and merged to master branch. Once merged, it’s available in the kuberntes repository.

ArgoCD detects the changes made in the kubernetes manifest repo, and sync the cluster with the desired state.It deploys the updated version of the application automatically ensuring that the cahnges are propagated across the cluster in a safe and controlled manner.

Using ArgoCD, we can also rollback if needed.

For example, this is the repo where all of our codes are saved

We have this deployment file here

Assuming ArgoCD is installed and let’s create a new application here

Press New App → fill details

Repository URL will have the repo link which ARGOCD will monitor all the time. Path will have the folder path where the yaml files are located in the repository.

Then Destination→ Cluster URL is where we include the link to cluster where we want to apply changes.

We can also use Edit as YAML button to make it easy

Here are the details for our case

Then save

Keep these changes

Then press Create button

Now if we check the pods, we can see that ArgoCD application is healthy and deployment is created (we had this yaml file in code)

Let’s make a change (replicas:3) to the deployment code in local system and push it to the github repo

As this code repo is connected to ArgoCD, it detects the changes and you can see status as Progressing now

Now if you check the pods, you can see 3 pods there

You can also see that ArgoCD UI

Done!!