Testing, CI Pipelines, and Mocking: A Week of Open Source Shenanigans

Introduction

This week in our Open Source Development course at Seneca, we were tasked with contributing tests to another student's repository and setting up a Continuous Integration (CI) pipeline for our own project. Peter and I swapped repositories and learned from each other's codebases. Peter worked on fixing an issue in my repo, GitHub Echo, while I tackled an issue in his repo, Gimme Readme.

We didn’t write a massive amount of tests - just enough to get our feet wet with testing in a language that wasn’t our own. I was working with Python and pytest, while Peter was using JavaScript and Jest. This gave us a chance to experience each other’s tools and setups.

The Jest Mocking Struggles

One part of this experience that really stood out to me was working with mocks in Jest. Testing with mocks can be tricky because it’s not just about checking if your functions run correctly, but also ensuring that certain behaviors are happening inside the code without actually executing them. It’s like testing an actor’s performance without watching the play.

Here’s an example from the issue I worked on in Peter’s repo, where I had to test a command-line utility with a --help flag. The goal was to ensure that when the flag is used, it correctly calls the help method on the commander program.

Here’s the code I wrote to achieve that:

import { jest } from '@jest/globals';
import handleHelpOption from '../../../src/option_handlers/handleHelpOption.js';
import commanderProgram from '../../../src/commander/commanderProgram.js';

describe('handleHelpOption', () => {
  test('should call help on program', () => {
    const helpSpy = jest.spyOn(commanderProgram, 'help').mockImplementation(() => {});

    handleHelpOption(commanderProgram);

    // Proves that passing the commanderProgram to handleHelpOption calls the help
    // method on the commanderProgram
    expect(helpSpy).toHaveBeenCalled();
    helpSpy.mockRestore();
  });
});

In this test, I used jest.spyOn() to create a spy function on commanderProgram.help, which simulates the call without actually running the method. I then checked if the help method was called by asserting expect(helpSpy).toHaveBeenCalled(). This is useful when we want to make sure certain functions or methods are invoked during execution without affecting the real logic. It’s an essential technique for unit testing, but it can also be a challenge to set up correctly.

The hardest part about testing with mocks is ensuring that the mocked functions are isolated and don’t interfere with the actual execution of the code. This is especially important when dealing with complex external systems or API calls, where running the actual function might be slow or result in side effects.

Setting Up My CI Pipeline

Another key part of the assignment this week was setting up a Continuous Integration (CI) pipeline for my project. The goal was to ensure that every push or pull request triggers tests and linting checks automatically. This way, we don't have to worry about forgetting to run tests manually, and we can catch errors early.

Here’s how I set up my CI pipeline using GitHub Actions. The pipeline is defined in a file called .github/workflows/ci.yml, and here’s the configuration I used:

name: CI Pipeline

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  workflow_dispatch:

jobs:
  code-lint:
    name: Lint with Ruff
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install ruff

      - name: Run Ruff
        run: ruff check .

  test:
    name: Run Tests
    runs-on: ubuntu-latest
    needs: ["code-lint"]

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install poetry
          poetry install

      - name: Set environment variables
        run: |
          echo "google_gemini_api_key=Sample" >> $GITHUB_ENV
          echo "github_api_token=Sample" >> $GITHUB_ENV
          echo "groq_api_key=Sample" >> $GITHUB_ENV

      - name: Run tests
        run: |
          poetry run run-tests

Explanation of the CI Pipeline

on section: This specifies when the workflow will trigger. It runs whenever there’s a push to the main branch, a pull request is made, or when triggered manually via the workflow_dispatch event.
jobs section: This is where we define the various tasks that will run during the pipeline. I have two main jobs:
1. code-lint job: This step runs the ruff linter, which checks for Python code style issues. The steps here involve checking out the code, setting up Python, installing dependencies, and running the linter.
2. test job: This job depends on the code-lint job. It runs only after the linting job is successful. It checks out the code, sets up Python, installs dependencies with poetry, sets environment variables for API keys, and finally runs the tests.
Environment Variables: I’ve added some environment variables for API keys that are required for the tests. These are set using the $GITHUB_ENV feature to ensure they’re available during the test run.

Why CI Pipelines Are Crucial

Setting up a CI pipeline like this ensures that my code is always tested in a clean environment before being merged into the main branch. It saves me from the fear of accidentally pushing untested code, and it automatically validates every change. This is a great way to ensure that the quality of the code remains high as the project grows.

Conclusion

This week was a great learning experience. Testing each other’s code gave me exposure to different testing tools and techniques, while setting up the CI pipeline helped me automate the testing process for my own project. It was a mix of overcoming challenges with mocks and diving deep into the mechanics of CI tools—both of which will be invaluable in future open-source projects.

Feel free to check out the issues we worked on: