From GitHub SaaS Project to CI/CD Automation

Introduction

Hello there! 👋 It's been a while since I last posted because of exams and a week of relaxation after coming back to my hometown for vacation. But in the meantime, I managed to do some useful work, and this past week, I got back on track with my DevOps learning. So for the past few weeks, I had been working on an AI GitHub SaaS project, and this week I got started with CI/CD processes using GitHub Actions and Jenkins.

GitHub SaaS Project

Previously, after finishing any of my projects, I wanted to document them for GitHub. I used to either create the README for that project on GitHub manually or give the context of what my project does to ChatGPT and just hope it would do a good job understanding the idea and context of the project. So then I thought, what if I create my own app to generate READMEs for my projects by using all of the context of my project directly from GitHub? From then on, I started researching about the GitHub API, how to get the source code of any project, and give it to any LLM to generate a perfect README for me.

Through the Octokit library, we can interact with the GitHub API and get the source code, but what if the source code for each file goes beyond the context limit of Gemini? So to solve this problem, I came to know about a method called "Prompt Chaining." In this method, first, we ignore the irrelevant files like package.json, node_modules, yarn.lock, bun.lock, .env, etc. Then we generate a summary for each relevant file. We store the summaries of all the relevant files in our database and then pass all those summaries as context to create the README.

While researching about this method, I was discussing with one of my friends about this, and he gave me a suggestion that if I can get the context of any project efficiently, I can do whatever I want with that data. And that sparked another idea - we can implement a Q&A-based feature where users can query anything about their codebase. But there's a problem with that too: how can we answer specific queries about anything from the codebase? How can we know which files are relevant for the query? The answer to that was Text Embedding.

We can create vector embeddings of the queries and the files of the codebase, and through matching them, we will get the relevant files and the required answer to the query using AI. I learned a few necessary things about Text Embedding and Vector databases and how I can use them in my project. For that, along with the generation of the summary of each file, we would also need to generate the embedding of the summary of each file and store it. Through them, we will get the answer to the query asked about the codebase. This could be very useful for junior developers or interns joining a new project or for open source projects where contributors want to have a thorough understanding of the codebase before contributing.

The tech stack I used for this project:

🔹 NextJS

🔹 TypeScript

🔹 TRPC

🔹 TailwindCSS

🔹 Shadcn

🔹 Clerk

🔹 Google Gemini API

🔹 Razorpay

🔹 PostgreSQL

🔹 Prisma

Feel free to check out the project and GitHub repository. Also, if you have any suggestions to improve the website, please do let me know.

GitHub link of the project: https://github.com/Akshansh029/Nebulo

Project link: https://nebulo-zeta.vercel.app

Getting Started with CI/CD

CI/CD with GitHub Actions

After completing the project and deploying it, this week I got back on track with DevOps. It was time to start one of the most important topics of DevOps - Continuous Integration and Continuous Delivery (CI/CD). As I had already studied about CI/CD and the principles used in it, I did a quick recap and wanted to start with the implementation part. I started with GitHub Actions.

GitHub Actions is a built-in CI/CD automation tool in GitHub. It lets you run workflows directly in your repository to automate tasks like:

• Building code

• Running tests

• Deploying to production

It works using YAML configuration files inside your repository.

The workflows are defined inside:

.github/
└── workflows/
    ├── ci.yml
    └── deploy.yml

Key components of a workflow are:

The steps defined in the .yml file are executed in virtual or physical machines called Runners. There are two types of runners which GitHub provides:

1. GitHub Hosted Runners (Shared) - These are provided by GitHub with pre-configured environments.

2. Self-hosted Runners - Users install and manage them and have complete control over hardware and OS.

I learned different concepts and components of GitHub Actions and how to use them in a workflow, such as using multiple jobs, environment variables, job concurrency, matrix strategy, etc. As practice, I wrote and used two workflows - one to build and test the source code of a Node application and another to build the Docker Image and store it in DockerHub.

integration.yml:

name: Node.js Build

on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [18.x, 20.x, 22.x]

    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: "npm"
      - name: Install dependencies
        run: npm install

      - name: Build command
        run: npm run build

  unit-tests:
    runs-on: ubuntu-latest
    needs: build
    strategy:
      matrix:
        node-version: [18.x, 20.x, 22.x]

    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: "npm"
      - name: Install dependencies
        run: npm install

      - name: Running unit test cases
        run: npm run test

deploy.yml:

name: Deploy on DockerHub

on:
  push:
    branches: ["main"]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Docker Build process
        run: docker build . -t akshansh29/next-js-app:latest

      - name: DockerHub login
        run: echo "${{secrets.DOCKERHUB_PASSWORD}}" | docker login -u ${{secrets.DOCKERHUB_USERNAME}} --password-stdin

      - name: Pushing image to DockerHub
        run: docker push akshansh29/next-js-app:latest

CI/CD with Jenkins

Jenkins is one of the most used DevOps tools to automate CI/CD workflows. It is an open-source automation server used to automate software development processes. Some core features of Jenkins are:

• Pipeline as Code using Jenkinsfile

• Extensible plugin ecosystem (e.g., Git, Maven, Docker, Slack, Kubernetes)

• Distributed builds using master-agent architecture

• Supports polling SCM or webhooks for triggering builds

• Can schedule jobs (cron-like syntax)

• Provides a dashboard and build history

To get started with Jenkins, I referred to the Jenkins docs for the installation guide. I created an EC2 instance to run the Jenkins server on it, SSH into it, and I copied the Linux commands to install Jenkins from the docs and set up the Jenkins server. I learned about the home directory of Jenkins, i.e., /var/lib/jenkins, where all the files of Jenkins are stored. Using the instance's IP address and port number, I accessed the Jenkins web page.

I also learned about Jobs - A Job in Jenkins is a task or project that Jenkins performs, such as pulling code from a repository, building a project, running tests, etc. It is of two types:

1. Freestyle project - A simple, GUI-based job configuration which is ideal for small or basic automation tasks.

2. Pipeline as Code - Jenkins Pipeline is a suite of plugins that support pipeline as code using a Jenkinsfile, which is a plain-text file checked into the source repository, enabling complex automation scenarios with full control.

Till now, I have learned how to create Freestyle projects and how to install plugins and tools in Jenkins. I learned how I can configure a freestyle project to install different tools like Git and Maven using shell commands and also by using the "Build Environment" section and selecting required tools. I configured a build job in which it should fetch the code from a GitHub repository, run the required tests, and build the artifact. All the relevant files and artifacts are stored in the workspace in the build section.

Resources I Used

1. GitHub Actions | KodeKloud

2. CI/CD Explained | TechWorld with Nana

3. Installation guide | Jenkins

4. DevOps Course | Udemy

Challenges I Faced

1️⃣ Too many requests to GitHub API

Solution: Collected only relevant files by filtering out using file extensions and implemented Exponential Backoff with retries, which is a strategy where you wait progressively longer before retrying a failed request. The delay increases exponentially (e.g., 500ms, 1000ms, 2000ms).

2️⃣ Error in testing the Vprofile using Maven due to a different JDK version

Solution: Installed the required JDK version.

3️⃣ Jenkins webpage access issue

Solution: Changed the inbound rule from my IP to every IPv4 address for testing and rebooted the instance.

What's Next?

I'll continue learning Jenkins, which is an essential tool that I want to master. I will learn about Pipeline as Code, practice by creating different pipelines, and make a project to create an end-to-end CI/CD pipeline.

Let's Connect!

🔗 My LinkedIn

🔗 My GitHub

If you have any recommended resources, better approaches to my challenges, or insights, I'd love to hear them! Drop your thoughts in the comments.

Have a wonderful day!