Breaking Down CI/CD Pipelines: What I Learned as a New Developer

Aryan JagdaleAryan Jagdale
5 min read

Why is it important?

Imagine Sarah, a developer at a growing startup. Every Thursday, her team would spend hours manually deploying new features - copying files, running tests, and praying nothing would break. One fateful deployment, a tiny typo slipped through, crashing their e-commerce site during peak shopping hours. They lost $20,000 in sales before finding and fixing the bug.

That's when Sarah championed implementing CI/CD pipelines. Now, her team's life is dramatically different. Code changes are automatically tested within minutes of submission. Deployments happen multiple times a day with just a click. When a bug recently tried to sneak in, the automated tests caught it instantly. The same error that once cost them thousands never made it to production.

This is the magic of CI/CD - it's like having a tireless quality guardian that ensures your code is always ready for prime time, protecting both developers' sanity and business value. The days of deployment anxiety are now just a distant memory for Sarah's team.

💡
It's important to remember that CI/CD is not the goal. The goal is better, faster software development with fewer preventable bugs and better team cooperation.

Concepts

GitHub Actions: The 3-Step Dance

  • The Trigger 🚀

    • Code push to main branch

    • Pull request creation

    • New issue opened

    • Or any defined event

  • The Workflow ⚙️

  • The Cleanup 🧹

Workflow

The core component of creating CI/CD pipelines with GitHub Actions is something called a Workflow. Workflows are process flows that you can set up in your repository to run automated tasks such as building, testing, linting, releasing, and deploying to name a few!

The hierarchy of a workflow looks as follows:

Workflow

  • Job

    • Step

    • Step

  • Job

    • Step

Job

Each workflow must specify at least one Job, which contains a set of Steps to perform individual tasks. The jobs will be run in parallel and the steps in each job will be executed sequentially.

Steps

Steps can vary from running a custom command to using pre-defined actions, thus the name GitHub Actions. You can create customized actions or use any actions published by the community, which are plenty, but let's get back to that later!

For GitHub to recognize your workflows, they must be specified in .github/workflows folder in your repository. Each Workflow is its own separate file which needs to be configured using the YAML data-serialization language.

YAML is a recursive acronym for "YAML Ain't Markup Language". As the name might hint its goal is to be human-readable and it is commonly used for configuration files. You will notice below that it is indeed very easy to understand!

Notice that indentations are important in YAML. You can learn more about the syntax here.

A basic workflow contains three elements

A basic workflow contains three elements in a YAML document. These three elements are:

  • name: Yep, you guessed it, the name of the workflow

  • (on) triggers: The events that trigger the workflow to be executed

  • jobs: The separate jobs that the workflow will execute (a basic workflow might contain only one job).

      name: Hello World!
      on:
        push:
          branches:
            - main
      jobs:
        hello_world_job:
          runs-on: ubuntu-20.04
          steps:
            - name: Say hello
              run: |
                echo "Hello World!"
    

When does GitHub trigger a workflow to be started…?

There are plenty of options to choose from, but generally speaking, you can configure a workflow to start once:

  • An event on GitHub occurs such as when someone pushes a commit to a repository or when an issue or pull request is created

  • A scheduled event, that is specified using the cron-syntax, happens

  • An external event occurs, for example, a command is performed in an external application such as Slack or Discord messaging app.

Anything that can go wrong…

You're about to deploy your latest code to production. Everything seems perfect until Murphy's Law kicks in - "Anything that can go wrong, will go wrong." As dramatic as it sounds, this age-old wisdom has humbled many developers.

Let me share why this matters in the real world.

The Deployment Drama: What Could Go Wrong?

  • Your trusty laptop decides it's the perfect time for a crash

  • Your internet connection plays hide and seek mid-deployment

  • That "foolproof" deployment script suddenly hits a snag

  • Your code works perfectly on your machine but throws a tantrum on the server

  • A user's request gets caught in the crossfire during deployment

The Golden Rules of Deployment

  1. Never Leave Your App Broken

  2. Silence Isn't Golden

    • Silent failures are like ninja bugs - deadly because you can't see them

    • If something goes wrong, you need to know ASAP

    • Imagine fixing a critical bug, but the deployment fails silently - yikes!

What does a good deployment system do?

Defining definitive rules or requirements for a deployment system is difficult, let's try anyway:

  • Our deployment system should be able to fail gracefully at any step of the deployment.

  • Our deployment system should never leave our software in a broken state.

  • Our deployment system should let us know when a failure has happened. It's more important to notify about failure than about success.

  • Our deployment system should allow us to roll back to a previous deployment

    • Preferably this rollback should be easier to do and less prone to failure than a full deployment

    • Of course, the best option would be an automatic rollback in case of deployment failures

  • Our deployment system should handle the situation where a user makes an HTTP request just before/during a deployment.

  • Our deployment system should make sure that the software we are deploying meets the requirements we have set for this (e.g. don't deploy if tests haven't been run).

In envisioning an ideal deployment system, speed and reliability stand out as paramount. The system should execute deployments rapidly while ensuring a seamless experience for users, completely avoiding any downtime.

This requirement emphasizes that while user requests are handled effectively during deployment, the core infrastructure remains uninterrupted, providing an impeccable service experience.

Through the exploration of various CI/CD pipeline strategies, we can harness these principles, fostering a robust framework that not only meets the demands of modern development but also paves the way for consistent delivery and innovation. With these insights, we can build a system that truly excels.

Below is my github repo, where I had built the CICD pipeline using references. Feel free to go through it.

References

2
Subscribe to my newsletter

Read articles from Aryan Jagdale directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Aryan Jagdale
Aryan Jagdale

Hello, I am Aryan. Currently I am a Final year Computer Engineering Student at VIT, Pune. I have experience in React , Redux with proficiency in web technologies like HTML5, CSS3, Typescript, building large-scale web applications, cross-browser platforms, and design constraints on the web, and building responsive user interfaces. Feel free to reach out to me for exciting opportunities.