Introduction to GitLab runners with setup in Depth

1. Introduction of GitLab

Kindly visit to my previous Blog for,

Basics of GitLab & Advance of GitLab

---

2. What are Runners in GitLab?

• GitLab Runner is a small lightweight program written in Go that runs Continuous Integration (CI) related jobs on your local machine and sends the results to GitLab for it to consider the changes.

• It is a single executable binary that can be installed on any major OS.

Types of Runners,

There are 3 types of runners in GitLab,

1. Instance runners

• Instance runners are available to every project in a GitLab instance.

• Use instance runners when you have multiple jobs with similar requirements. Rather than having multiple runners idling for many projects, you can have a few runners that handle multiple projects.

2. Group runners

• Use group runners when you want all projects in a group to have access to a set of runners.

• Group runners process jobs by using a first in, first out queue.

3. Project runners

Use project runners when you want to use runners for specific projects. For example, when you have:

• Jobs with specific requirements, like a deploy job that requires credentials.

• Projects with a lot of CI activity that can benefit from being separate from other runners.

---

3. Features of GitLab runners?

GitLab Runner has the following features.

• Run multiple jobs concurrently.

• Use multiple tokens with multiple servers (even per-project).

• Limit the number of concurrent jobs per-token.

• Jobs can be run:

  • Locally.

  • Using Docker containers.

  • Using Docker containers and executing job over SSH.

  • Using Docker containers with autoscaling on different clouds and virtualization hypervisors.

  • Connecting to a remote SSH server.

• Is written in Go and distributed as single binary without any other requirements.

• Supports Bash, PowerShell Core, and Windows PowerShell.

• Works on GNU/Linux, macOS, and Windows (pretty much anywhere you can run Docker).

• Allows customization of the job running environment.

• Automatic configuration reload without restart.

• Easy to use setup with support for Docker, Docker-SSH, Parallels, or SSH running environments.

• Enables caching of Docker containers.

• Easy installation as a service for GNU/Linux, macOS, and Windows.

• Embedded Prometheus metrics HTTP server.

• Referee workers to monitor and pass Prometheus metrics and other job-specific data to GitLab.

---

4. Runners execution flow

This diagram shows how runners are registered and how jobs are requested and handled. It also shows which actions use registration, authentication, and job tokens.

Terms related to GitLab Runner,

• GitLab Runner: The application that you install that executes GitLab CI jobs on a target computing platform.

• runner configuration: A single [[runner]] entry in the config.toml that displays as a runner in the UI.

• runner manager: The process that reads the config.toml and runs all the runner configurations concurrently.

• runner: The process that executes the job on a selected machine. Depending on the type of executor, this machine could be local to the runner manager (shell or docker executor) or a remote machine created by an autoscaler (docker-autoscaler or kubernetes).

• machine: A virtual machine (VM) or pod that the runner operates in. GitLab Runner automatically generates a unique, persistent machine ID so that when multiple machines are given the same runner configuration, jobs can be routed separately but the runner configurations are grouped in the UI.

---

5. Setup of GitLab runners

We have running pipeline from previous blog,

To setup runner,

Go to Setting of an project -> CICD

• Select runners tab and expand it.

• Click on "New Project Runner"

• Give tags for your runner : dev (dev, prod, test, etc)

• Click on "Create runner"

• You will get below screen.

• Choose platform of your runner : Linux* / Windows / MacOS

Here is an important note that, we need to install runner first before registration of runner.

• Installing runner in Linux,

• Copy all the commands given and create an script called runner.sh.

• Install, curl command.

• Note, user privilege must be root.

# Download the binary for your system
sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64

# Give it permission to execute
sudo chmod +x /usr/local/bin/gitlab-runner

# Create a GitLab Runner user
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

# Install and run as a service
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start

• Runner installation is in progress,

• Runner is now successfully installed.

• Now, register runner with our project.

• Copy token given and paste it in runner.

gitlab-runner register  --url https://gitlab.com  --token glrt-6cHadoos22kj1R8b****

• Now, press ENTER

• Give name to runner : dev-runner

• Select executer : shell

• Click ENTER

• Finally runner is registered successfully.

• Check status of runner, it requires root priviledges.

• Runner is visible in project.

• Run runner,

sudo gitlab-runner run

• Runner is ready to execute jobs,

This is how we can setup runners on GitLab.

---

Jobs are running on SaaS runners by GitLab,

• Run jobs on the project runner, edit .gitlab-ci.yml to enable.

stages: # Define your stages first to configure pipeline and its an mandatory steps.
    - build
    - test
    - push
    - deploy

build_job:
    stage: build 
    script:
        - echo "This $CI_PROJECT_NAME build is done using command <docker build -t .>"
        - echo "This job is build by $CI_COMMIT_AUTHOR"
        - mkdir -p logs/
        - echo "This are my test results" > logs/app.log
    artifacts: # To store artifacts
        paths:
            - logs/ # Directory to store
        expire_in: 1 week # Logs should be expire in 1 week
    tags:
        - dev

test_job:
    stage: test
    script:
        - echo "This is testing stage of our docker build"

push_job:
    stage: push
    script: 
        - echo "Pushing image to dockerHub"
    tags:
        - dev

deploy_job:
    stage: deploy
    script: 
        - echo "Deploying to EC2"
    tags:
        - dev

dev_test_job: 
    stage: test # Stage should be "test" so that is will be parallel job.
    script:
        - echo "This is tested for dev"
        - echo "This is an log from $CI_JOB_STAGE" >> logs/app.log
    artifacts: # To store artifacts
        paths:
            - logs/ # Directory to store
        expire_in: 1 week # Logs should be expire in 1 week

prod_test_job:
    stage: test # Stage should be "test" so that is will be parallel job.
    script:
        - echo "This is tested for prod"

tags:
   - dev # Tag name of runner we configured

Above we have configured, dev runner for Jobs build, push, deploy.

• Commit changes.

-> Pipeline jobs running execution scenario,

• Status of build job, executed on dev runner.

• Status of test job, executed on SaaS runner.

• Viewing project artifacts on runner called dev-runner,

• Go to runner's home directory,

$

This is how we can use runners to build our Jobs in GitLab CICD.

---

Connect With Me

LinkedIn | GitHub

Follow me for more amazing content :)