Testing and Continuous Integration

Previously for CI/CD I have used CircleCI, TravisCI and other platforms (like Bamboo). For this project I will use GitHub Actions to see if that can do everything I need.

In the Actions tab of my GitHub repository is the option to add a new workflow. There is a default set up for Ruby on Rails so I'll start with that.

I've tweaked the file that is generated so that minitest is run in the tests job and the results of the SimpleCov code coverage are uploaded to CodeClimate. It is also using a test environment I created in GitHub with a single secret for the CC_TEST_REPORTER_ID.

.github/workflows/rubyonrails.yml

# This workflow uses actions that are not certified by GitHub.  They are
# provided by a third-party and are governed by separate terms of service,
# privacy policy, and support documentation.
#
# This workflow will install a prebuilt Ruby version, install dependencies, and
# run tests and linters.
name: "CI"
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
jobs:
  test:
    runs-on: ubuntu-latest
    environment: test
    services:
      postgres:
        image: postgres:14-alpine
        ports:
          - "5432:5432"
        env:
          POSTGRES_DB: rails_test
          POSTGRES_USER: rails
          POSTGRES_PASSWORD: password
    env:
      RAILS_ENV: test
      DATABASE_URL: "postgres://rails:password@localhost:5432/rails_test"
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      # Add or replace dependency steps here
      - name: Install Ruby and gems
        uses: ruby/setup-ruby@8f312efe1262fb463d906e9bf040319394c18d3e # v1.92
        with:
          bundler-cache: true
      - name: Set up Chrome
        uses: browser-actions/setup-chrome@latest
        with:
          chrome-version: stable
      - name: Download chrome driver
        uses: nanasess/setup-chromedriver@v1.0.7
      - name: Launch chrome driver
        run: |
          export DISPLAY=:99
          chromedriver --url-base=/wd/hub &
          sudo Xvfb -ac :99 -screen 0 1280x1024x24 > /dev/null 2>&1 & # optional
      # Add or replace database setup steps here
      - name: Set up database schema
        run: bin/rails db:schema:load
      # Add or replace test runners here
      - name: Run Tests and Upload Code Coverage
        uses: paambaati/codeclimate-action@v3.0.0
        env:
          CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
        with:
          coverageCommand: bin/rails test:all
          debug: true
      - name: Upload test failures
        uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: Test Failures
          path: /home/runner/work/catalogue_cleanser/catalogue_cleanser/tmp/screenshots
          retention-days: 5

  lint:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Install Ruby and gems
        uses: ruby/setup-ruby@8f312efe1262fb463d906e9bf040319394c18d3e # v1.92
        with:
          bundler-cache: true
      # Add or replace any other lints here
      - name: Lint Ruby files
        run: bundle exec rubocop --parallel

On an initial run this has failed though with the following error. Luckily it has the appropriate instructions for fixing it.

Gemfile.lock

PLATFORMS
  x86_64-darwin-20

% bundle lock --add-platform x86_64-linux
% bundle lock --add-platform ruby

Gemfile.lock

PLATFORMS
  ruby
  x86_64-darwin-20
  x86_64-linux

I also needed to add the following section to the .github/workflows/rubyonrails.yml file so that Selenium could launch Chrome to run the system tests in the scaffolded application.

      - name: Set up Chrome
        uses: browser-actions/setup-chrome@latest
        with:
          chrome-version: stable
      - name: Download chrome driver
        uses: nanasess/setup-chromedriver@v1.0.7
      - name: Launch chrome driver
        run: |
          export DISPLAY=:99
          chromedriver --url-base=/wd/hub &
          sudo Xvfb -ac :99 -screen 0 1280x1024x24 > /dev/null 2>&1 & # optional

The other change was to application_system_test_case.rb to indicate I want the headless version of Chrome.

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400]
end

In order to get the code coverage uploaded to CodeClimate the following needs to be added to the .github/workflows/rubyonrails.yml file.

      - name: Run Tests and Upload Code Coverage
        uses: paambaati/codeclimate-action@v3.0.0
        env:
          CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
        with:
          coverageCommand: bin/rails test:all
          debug: true

The .simplecov file is where SimpleCov configuration is placed. The filter The minimum coverage is what percentage of lines are covered before considered passing. 90% is going to be my goal here but at the moment I'm only getting to around 87% due to some controller responses not being tested yet.

SimpleCov.start :rails do
  minimum_coverage 80
  maximum_coverage_drop 2
end

And in the test_helper.rb file I tell SimpleCov to execute by requiring 'simplecov'. It also appears that SimpleCov does not like the new parallel testing setting so that needs to be disabled.

class ActiveSupport::TestCase
  # Run tests in parallel with specified workers
  # DOES NOT WORK WITH SimpleCov
  # parallelize(workers: :number_of_processors)

  # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
  fixtures :all

  # Add more helper methods to be used by all tests here...
end

If there are failures in the system tests these will produce screenshots to assist with troubleshooting what the problem is. By adding this step to the .github/workflows/rubyonrails.yml file those screenshots will be saved as an artifact that can be downloaded.

      - name: Upload test failures
        uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: Test Failures
          path: /home/runner/work/catalogue_cleanser/catalogue_cleanser/tmp/screenshots
          retention-days: 5

Here is a screenshot of a failed run with a link to the artifact.

I can also add the status of the workflow to the README.md so it is clear whether all the tests and linting are passing currently or not. The test coverage statistic is now also correctly returning the coverage result.

![GitHub Workflow](https://github.com/andrewfoster73/catalogue_cleanser/actions/workflows/rubyonrails.yml/badge.svg)

Useful links:

https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment

https://docs.github.com/en/actions/security-guides/encrypted-secrets

https://github.com/actions/upload-artifact

https://www.moncefbelyamani.com/understanding-the-gemfile-lock-file/#platforms

https://github.com/simplecov-ruby/simplecov/issues/718