Setup RSpec Tests in Rails with Gitlab CI

At Zero Config Rails, I (Prabin) am constantly working on automating configurations and boring setups like “Setup RSpec Tests in Rails with Gitlab CI”.

CI integrations helps in improving the code quality in projects. It can help in automating code reviews for linting and standard practices as well as for running tests to check if code change breaks any existing functionalities.

Today we will look at adding configurations to Gitlab CI for running RSpec tests in our Rails application.

For the detailed list of configurations, you can visit Gitlab CI Generator.

Now without further ado, let’s jump right into setting up Gitlab CI for RSpec and run those tests in CI.

Assumption

• You have basic Gitlab CI configurations ready i.e. .gitlab-ci.yml exists in your project. If it doesn't, you can refer to my other article Integrate Pronto with Gitlab for Rails App.

• You are using PostgreSQL in your app though with minimal changes it should also work for any other databases.

Tested and working in

• Ruby 3.3.0

• Rails 7.1.3

• rspec-rails 6.1.1

• selenium-webdriver 4.18.1

Configure Gitlab CI Variables

Firs of all, we need to add some configurations required by the CI to run tests. This should be done over at Gitlab.

Add variable for storing environment variables

I normally use Figjam which is a maintained version of the popular Figaro gem for storing environment variables which uses config/application.yml but just the plain .env file using dotenv gem is also very popular. Anyway, just copy the content from whatever you are using and paste it inside the Value for this new variable.

You can visit the official documentation to learn about setting up variables for Gitlab CI. You have to go to your project's setting in Gitlab and configure these in CI/CD variables.

Create a new variable for storing content in your config/application.yml:

1. Type: File

2. Flags

   Uncheck all checklists here i.e. Protect variable, Mask variable and Expand variable reference

3. Description

   You can add “Environment Variables“ but it's optional and you can skip this as "Key" (just below this) is already clear enough on what this variable is storing.

4. Key: env

In "Value", add the copied content from your env file.

NOTE: Make sure to only copy what is under "test" block or ".env.test", you don’t want to add production variables here!

Add variable for MASTER_KEY

Rails comes with config/credentials.yml.enc for storing secrets, we generally also use ENV variables for this but since Rails credentials is the default, we will also look at how to configure those.

To decrypt credentials file, you need MASTER_KEY. If you have generated multiple credentials file per environment then you might have multiple keys like master.key, staging.key, production.key, etc..

Create a new variable for storing content in the “.key” file that can decrypt secrets configured for the test environment; normally this will be inside the config/master.key:

1. Type: Variable (Default)

2. Flags

   Uncheck all checklists here i.e. Protect variable, Mask variable and Expand variable reference

3. Description

   Optional. You can leave it blank.

4. Key: MASTER_KEY

And in Value, add the content copied from the master.key

Add database.yml.ci file

It's not considered a good practice to use config/database.yml file for the CI so we will instead create a new file config/database.yml.ci and add configurations required to run tests inside.

You can visit the official documentation to learn about setting up variables. You have to go to your project's setting in Gitlab and configure these in CI/CD variables.

After creating the file, add the following:

test:
  adapter: postgresql
  encoding: unicode
  host: postgres
  database: ci_db
  username: postgres
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

For username it should be “postgres” which is the default user that gets created when postgres service/docker is created hence it doesn’t ask password or tries to authenticate user. You might get an error otherwise because no other users will have been created in postgres at this point: `Please check your database configuration to ensure the username/password are valid`.

For host make sure to use "postgres" instead of “localhost”. For MySQL, you will have to use "mysql" as said in the official documentation:

The service container for MySQL is accessible under the hostname mysql. To access your database service, connect to the host named mysql instead of a socket or localhost.

SQLite doesn’t need any host configurations but other configurations will most probably vary.

Configure Capybara with Selenium

We will configure Selenium with Chrome to be used both in CI and Local with Headless mode (by default) while also allowing to run in the browser if needed for debugging.

Create a new file "spec/support/capybara.rb" and add the following code:

Capybara.register_driver :selenium_chrome_custom do |app|
  options = Selenium::WebDriver::Chrome::Options.new

  options.add_argument("--headless=new") unless ENV["SELENIUM_HEADFUL"]

  options.add_argument("--window-size=1400,1400")
  options.add_argument("--no-sandbox")
  options.add_argument("--disable-dev-shm-usage")

  remote_url = ENV["SELENIUM_REMOTE_URL"]

  if remote_url
    Capybara::Selenium::Driver.new(
      app,
      browser: :remote,
      url: remote_url,
      options:
    )
  else
    Capybara::Selenium::Driver.new(app, browser: :chrome, options:)
  end
end

RSpec.configure do |config|
  config.before(:each, type: :system, js: true) do
    # Make the test app listen to outside requests, required for the remote Selenium instance
    Capybara.server_host = "0.0.0.0"
    Capybara.server_port = 3000

    if ENV["SELENIUM_REMOTE_URL"]
      # Use the application container's IP instead of localhost so Capybara knows where to direct Selenium
      ip = Socket.ip_address_list.detect(&:ipv4_private?).ip_address
      Capybara.app_host = "http://#{ip}:#{Capybara.server_port}"
    end

    driven_by :selenium_chrome_custom
  end
end

Explanation

Let's look at what each of the code block above is doing.

Custom Selenium Chrome driver

Capybara.register_driver :selenium_chrome_custom

Since existing Selenium Drivers don't provide the custom options we want, we are creating a new driver selenium_chrome_custom which will handle Remote/Local connection as well as Headless/Headful mode.

Options

• --window-size=1400,1400

  Set the window size to 1400x1400 pixels. This is a reasonable size without being too large, but you can set it to whatever you like. This mostly impacts the size of debugging screenshots, but some tests may fail if you ask Capybara to click on an element which is not currently visible on the page.

• --no-sandbox

  Disables Chrome’s sandbox functionality because it has an issue with Docker version 1.10.0 and later.

• --disable-dev-shm-usage

  The "/dev/shm" shared memory partition is too small on many VM environments which will cause Chrome to fail or crash so we are disabling it.

• --headless=new

  Enable Chrome’s headless mode which will run Chrome without a UI.

  SELENIUM_HEADFUL will control this option. In development, you may want to run Chrome and see what's happening in the browser; you can do so by running tests with SELENIUM_HEADFUL=true bundle exec rspec spec/system.

  We will see list of other commands to run system tests at the end of this explanation section in a bit.

Some guides may suggest using the --disable-gpu flag, but this is no longer necessary on any operating system.

This explanation was shamelessly copied from Remote Selenium WebDriver servers with Rails, Capybara, RSpec, and Chrome 🙈.

Selenium remote URL

remote_url = ENV["SELENIUM_REMOTE_URL"]

Remote option is required mostly for CI but you can also test it out in local by running the Selenium Docker image e.g. with SELENIUM_REMOTE_URL=http://localhost:4444/wd/hub bundle exec rspec spec/system

Remote option is controlled by SELENIUM_REMOTE_URL which needs to be passed when running tests as seen above.

Another configuration related to the remote is the use of browser: :remote inside Capybara::Selenium::Driver.new which tells Capybara to run tests in remote Chrome browser instead of local one.

Capybara server and app host

RSpec.configure do |config|
  config.before(:each, type: :system, js: true) do
    # Make the test app listen to outside requests, required for the remote Selenium instance
    Capybara.server_host = "0.0.0.0"
    Capybara.server_port = 3000

    if ENV["SELENIUM_REMOTE_URL"]
      # Use the application container's IP instead of localhost so Capybara knows where to direct Selenium
      ip = Socket.ip_address_list.detect(&:ipv4_private?).ip_address
      Capybara.app_host = "http://#{ip}:#{Capybara.server_port}"
    end

    driven_by :selenium_chrome_custom
  end
end

server_host and app_host are required for Capybara to know how it can call driver in the Remote Server.

This piece of code was extracted from the official Rails Documentation.

Commands to run tests

Lastly, let's see various commands we can use to run system tests.

• Run in headless mode (default): bundle exec rspec spec/system

• Run in headful mode: SELENIUM_HEADFUL=true bundle exec rspec spec/system

• Run in headless mode inside external docker image in local: SELENIUM_REMOTE_URL=http://localhost:4444/wd/hub bundle exec rspec spec/system

For CI, default command bundle exec rspec spec/system will work. But SELENIUM_REMOTE_URL will be http://selenium:4444/wd/hub and it will be passed an Environment Variable instead. We will look at how to do that next.

Update .gitlab-ci.yml to run all tests

We will be adding code to enable all the following tests and you can choose to pickup or ignore as per your requirement:

• Unit and Integration tests (Model, Requests, Authorization, Services etc.) which don't require us to start browser

• System Tests where we will start the Chrome browser and run tests inside it

Update your .gitlab-ci.yml with the configurations given below. Most of the configurations are accompanied by explanation, you can find clean configuration without comment at the end of the blog in the section "Final .gitlab-ci.yml"

# change to the ruby version your application uses
image: ruby:3.3.0

variables:
  MASTER_KEY: $MASTER_KEY

# explanation in next section
cache:
  paths:
    - vendor/
    - node_modules/
    - yarn.lock

stages:
  - test

# base configuration required for running tests
.base_db:
  # add-on docker images required for running tests
  services:
    - postgres:latest
  variables:
    # set Rails environment so we don't have to prefix each command with RAILS_ENV=test
    RAILS_ENV: test
    # Postgres runs in a separate docker image and requires authentication to connect. Disabling that here by using "trust" so it doesn't ask for authentication
    POSTGRES_HOST_AUTH_METHOD: trust
  before_script:
    # use same bundler version that was used in bundling the Gemfile
    - gem install bundler -v "$(grep -A 1 "BUNDLED WITH" Gemfile.lock | tail -n 1)" --no-document
    # install all gems to "vendor" folder which helps in caching of gem installation in between the execution of CI jobs
    - bundle config set --local path 'vendor'
    # install "nodejs" required for yarn and "cmake" required for pronto
    - apt-get update -qq && apt-get install -y -qq nodejs cmake
    # install gems in parallel, nproc returns the number of available processors
    - bundle install --jobs $(nproc)
    # install yarn
    - curl -o- -L https://yarnpkg.com/install.sh | bash
    # Make yarn available in the current terminal
    - export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"
    - yarn install
    - cp config/database.yml.ci config/database.yml
    # 👋 config/application.yml can be different for you. For e.g. if you are using ".env" then this content will be `cat $env > .env`
    - cat $env > config/application.yml
    - bundle exec rails db:test:prepare

unit_and_integration_tests:
  # reuse all configurations defined in .base_db above
  extends: .base_db
  stage: test
  # run this job only when merge requests are created, updated or merged
  only:
    - merge_requests
  script:
    # run all tests except system tests
    - bundle exec rspec --exclude-pattern "spec/system/**/*.rb"

system_tests:
  extends: .base_db
  stage: test
  services:
    # need to declare postgres again because "services" key will override the one defined in .base_db
    - postgres:latest
    # Docker image for Selenium with Chrome so test can run inside the browser
    - name: selenium/standalone-chrome:latest
      alias: selenium
  variables:
    RAILS_ENV: test
    POSTGRES_HOST_AUTH_METHOD: trust
    # Location of the selenium docker image. "selenium" is an alias, you can also use http://selenium-standalone-chrome:4444/wd/hub or selenium__standalone-chrome (commonly seen in other guides)
    SELENIUM_REMOTE_URL: http://selenium:4444/wd/hub
  # store necessary files and folders in case of test failure for debugging the error
  artifacts:
    when: on_failure
    paths:
      - log/test.log
      - tmp/screenshots/
    expire_in: 1 week
  only:
    - merge_requests
  script:
    - bundle exec rspec spec/system

Explanation

Let's look at some configurations where explanation was missing and would be lengthy to add there.

cache

cache:
  paths:
    - vendor/
    - node_modules/
    - yarn.lock

This tells Gitlab CI to cache vendor folder where we are storing all our gems, node_modules where all JS packages are stored, yarn.lock which stores the information about installed packages with their versions.

Storing all these folders and files speed up the CI in subsequent runs. bundle install and yarn install will only install new packages that are not already inside the cache.

stages

stages:
  - test

Stages define when to run the jobs. For example, stages that run tests after stages that runs linting on new changes.

If you also have linting and continuous deployment configured then stages could look like this:

stages:
  - lint
    - test
    - staging_deploy
    - production_deploy

Jobs are run in the same order as configured here i.e. linting will run first then test and lastly deployments.

.base_db

This configuration is used by all jobs that require database access. All common configurations for such jobs are extracted here.

services are add-on docker images and provide capabilities like database, redis, selenium drivers, etc.

variables are environment variables used by Rails.

before_script runs before the script so anything that needs to be pre-configured can be added here.

unit_and_integration_tests

extends will extend the configurations defined in the .base_db and use those configurations for this job.

stage tells this job at what stage to run. Depending on stages defined just above this job configuration.

script are the series of command to execute for this job. We are running all tests except system tests by using the rspec command helper --exclude-pattern "spec/system/**/*.rb

system_tests

selenium/standalone-chrome:latest configures the docker image for Selenium with Chrome with the latest version. One thing to note is that latest version can be unstable; one time I had to spend 6+ hours in debugging just to find out the latest version "123.0" had some issues and Chrome browser was not starting. You can try with lower versions in that case by adding the version number at the end instead of “latest” e.g. selenium/standalone-chrome:122.0 and verify if it resolves the issue.

artifacts is used to store necessary files and folders in case of test failure. This helps us in debugging failing tests when needed. We are storing test log files for this purpose.

Final .gitlab-ci.yml

If you also have Pronto or any other linter configured in CI then your final file could look like this:

image: ruby:3.3.0

variables:
  MASTER_KEY: $MASTER_KEY

cache:
  paths:
    - vendor/
    - node_modules/
    - yarn.lock

stages:
  - test

.base_db:
  services:
    - postgres:latest
  variables:
    RAILS_ENV: test
    POSTGRES_HOST_AUTH_METHOD: trust
  before_script:
    - gem install bundler -v "$(grep -A 1 "BUNDLED WITH" Gemfile.lock | tail -n 1)" --no-document
    - bundle config set --local path 'vendor'
    - apt-get update -qq && apt-get install -y -qq nodejs cmake
    - bundle install --jobs $(nproc)
    - curl -o- -L https://yarnpkg.com/install.sh | bash
    - export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"
    - yarn install
    - cp config/database.yml.ci config/database.yml
    - cat $env > config/application.yml
    - bundle exec rails db:test:prepare

unit_and_integration_tests:
  extends: .base_db
  stage: test
  only:
    - merge_requests
  script:
    - bundle exec rspec --exclude-pattern "spec/system/**/*.rb"

system_tests:
  extends: .base_db
  stage: test
  services:
    - postgres:latest
    - name: selenium/standalone-chrome:latest
      alias: selenium
  variables:
    RAILS_ENV: test
    POSTGRES_HOST_AUTH_METHOD: trust
    SELENIUM_REMOTE_URL: http://selenium:4444/wd/hub
  artifacts:
    when: on_failure
    paths:
      - log/test.log
      - tmp/screenshots/
    expire_in: 1 week
  only:
    - merge_requests
  script:
    - bundle exec rspec spec/system

Conclusion

Phew, that was a lot of configurations and explanation. And you can automate all of this with just a single command from Zero Config Rails in near future, stay tuned!

With this, your Rails app now has all type of tests running in the Gitlab CI so you can now merge changes without any worry for them breaking the production application.

Thank you for reading. Happy coding!

References

• How services are linked to the Job (Gitlab)

• Gitlab CI Config for System Tests with Minitest (Github Gist)

• Remote Selenium WebDriver servers with Rails, Capybara, RSpec, and Chrome

• System Testing (Official Rails Documentation)

Image Credits:

• Cover Image by Jens Freudenau on Unsplash