Deploying an application using Docker, EKS and Argo CD

Hello everyone, Hope you are doing well. Today I'm implementing a project which is a part of 10WeeksofCloudOps, project 4. In this project, we are going to deploy an application based on the concept of GitOps, in our case GitHub is considered as the single source of truth.

A huge shout out to The CloudOps Community and Piyush Sachdeva, who has implemented this project on his YouTube channel. Here is the diagram that shows us how the project has been implemented

First of all, we will configure the AWS CLI for our visual studio code following this guide:

https://docs.aws.amazon.com/toolkit-for-vscode/latest/userguide/welcome.html

Configure the AWS CLI for the root user using the

aws configure command, in the blog we are going to use the following resources using the following custom terraform modules.

• EKS

• NatGateway

• IAM

• Node Group

• VPC

As a part of the project first create an S3 bucket and a Dynamo DB table to store the tfstate file of our project. Make sure the Public access to the S3 bucket has been blocked, and enable the bucket versioning.

Create a DynamoDB table and make sure the partition key is set to LockID String type.

Here is the source code for application in the git repository:

https://github.com/YaswanthRayana1/AppCode

To deploy the application on an EKS cluster we need the infrastructure, clone the git repository locally and make necessary changes to the backend.tf file

https://github.com/YaswanthRayana1/10weeksofcloudops-demo4-tf

Kube manifest git repo

https://github.com/YaswanthRayana1/kube_manifest

Clone both repositories, remove the Git remote for the application code folders and add your Git remote by following this guide

https://stackoverflow.com/a/44076938/22772068 and upload the code to your git repository.

To start first, change the

cd 10weeksofcloudops-demo4-tf/ToDo-App/

Use any of the code editors

Change the Bucket name to the S3 bucket you created in AWS S3.

terraform {
  backend "s3" {
    bucket = "week4"
    key    = "backend/ToDo-App.tfstate"
    region = "us-east-1"
    dynamodb_table = "week4"
  }
}

• We also need to create a terraform.tfvars file that contains the variables like public and private subnet CIDRs

REGION          = "us-east-1"
PROJECT_NAME    = "ToDo-App"
VPC_CIDR        = "10.0.0.0/16"
PUB_SUB1_CIDR   = "10.0.1.0/24"
PUB_SUB2_CIDR   = "10.0.2.0/24"
PRI_SUB3_CIDR   = "10.0.3.0/24"
PRI_SUB4_CIDR   = "10.0.4.0/24"

Since we already have the terraform modules for all the resources we are creating on the AWS, we can create them as follows:

terraform init will initialize the terraform dependencies in the 10weeksofcloudops-demo4-tf folder

Once dependencies have been initialized, use the

terraform plan to check the resources that are going to be created,

terraform apply and use this command to create the infrastructure.

• Once the infrastructure has been created, verify the resources in the AWS console.

  aws eks update-kubeconfig --region us-east-1 --name ToDo-App

Use this command to add the kube config file of the EKS cluster we created to our kube context, so now all the kubectl commands we use will point to the cluster we have created using the terraform apply.

• In this project we are using Circle CI for Continuous Integration and Argo CD for Continuous Delivery. Use GitHub to sign up for CircleCI.

Continuous Integration

In the app code repository create a folder .cicleci/config.yml and use the following code to build, push and update the manifest of the deployment.yml file.

version: 2.1

jobs:
  build_and_push:
    docker:
      - image: cimg/node:20.3.1
    steps:
      - checkout
      - setup_remote_docker
      - run:
          name: Lets build and push image
          command: |
            version="build-$CIRCLE_BUILD_NUM"
            echo $version
            docker build -t yaswanthrayana/todo-app:$version .
            echo $DOCKER_PASSWORD | docker login -u $DOCKER_USERNAME --password-stdin
            docker push yaswanthrayana/todo-app:$version

  Update_manifest:
    docker:
      - image: cimg/base:2023.06
    steps:
      - checkout
      - setup_remote_docker
      - run:
          name: Updating Manifest file 
          command: |
            TAG=$CIRCLE_BUILD_NUM
            ((TAG--))
            git clone https://github.com/YaswanthRayana1/kube_manifest.git
            git config --global user.email "yaswanthrayana91221@gmail.com"
            git config --global user.name "YaswanthRayana1"
            pwd
            cd kube_manifest
            pwd
            echo $TAG
            sed -i "s/build-.*/build-$TAG/g" manifest/deployment.yaml
            cat manifest/deployment.yaml
            git add .
            git commit -m "new build with imgTag build-$TAG"
            git config credential.helper 'cache --timeout=120'
            git push -q https://$GITHUB_PERSONAL_TOKEN@github.com/YaswanthRayana1/kube_manifest.git main 
workflows:
  GitOpsflow:
    jobs:
      - build_and_push
      - Update_manifest:
          requires:
            - build_and_push

Make necessary changes to the above file such as the GitHub username, Docker hub username, Email id etc.,

• Create a personal access token in GitHub to use in the CircleCI to trigger the pipeline in the CircleCI

  https://github.com/settings/tokens?type=beta

• Create a Fine-grained token and give admin access to the git repository that contains the Application code along with the .cicleci/config.yml and save it to configure the CircleCI project settings.

• Once the pipeline is configured, CircleCI should build a docker image and push it to the DockerHub.

Continuous Deployment

Run the following commands to install ArgoCD

kubectl create namespace argocd

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.4.7/manifests/install.yaml

This deploys the ArgoCD application controller and other required objects in the argocd namespace and to manage the application.

We need to port forward the ArgoCD server to access the GUI,

kubectl port-forward svc/argocd-server 8080:443 -n argocd

Open a new terminal to get the ArgoCD secret, get the ArgoCd secret on the argocd namespace.

kubectl get secrets -n=argocd this gives us the secrets that have been deployed in the argocd namespace.

The initial admin secret is in base64 encoded to get the secret

kubectl get secret argocd-initial-admin-secret -n argocd -o yaml

Copy the base 64 encoded password and decode the secret using the echo command:

echo <base-64-encoded-password> | base64 --decode

Use the password to log into the ArgoCD dashboard.

Create a argo-manifest.yml file that contains the GitHub URL for the Deployment and Service files, ArgoCD pulls the new docker image and deploys it in the destination myapp namespace.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: todo-app-argo
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/YaswanthRayana1/kube_manifest
    targetRevision: HEAD
    path: manifest
  destination: 
    server: https://kubernetes.default.svc
    namespace: myapp
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
    automated:
      selfHeal: true
      prune: true

Update the GitHub URL for the repoURL in the manifest file and apply the manifest file

kubectl apply-f argo-manifest.yml

This should pull the configuration from our git repository and apply the changes to the Kubernetes myapp namespace.

So Here comes the end of this blog,

Every time there is a change in the appcode Git repository the CircleCI pulls the code and builds the docker image, pushes it to the DockerHub and updates the build number on the deployment.yml Since our ArgoCD is watching the kube_manifest git repository, ArgoCD pulls the changes and applies them to the myapp namespace.

Thank you for following along with this guide creating a CICD pipeline using CircleCI and deploying the application on an EKS cluster using ArgoCD to manage the Continuous Deployment. Hope this blog helps, for any questions feel free to comment. I will try to resolve the errors.