GitLab CI Pipelines with 1Password and private Flutter packages

frankylee kellyfrankylee kelly
8 min read

Table of contents

Imagine you're used to using a specific tool (GitHub) and suddenly you're required to use a new tool (GitLab) so you have to figure out a whole new system for your CI pipeline. We've all been there.

This guide will walk you through setting up a GitLab CI pipeline for a Flutter application that uses app Flavors, 1Password for secrets management, fvm for Flutter SDK versioning, and private GitLab repositories. If you want to skip ahead, jump to the bottom for the complete .gitlab-ci.yml.

💡
This guide is only intended to setup the Flutter application to run tests for the GitLab continuous integration pipeline. It will not cover the continuous deployment pipeline.

Prerequisites

Before we continue, I'm going to make the following assumptions:

  • You've already setup app Flavors for your Flutter project following the official documentation. If it's not relevant to you, that's fine—it's not necessary in order to follow this guide.

  • You use 1Password to manage your secrets and need the CLI tools to retrieve said secrets and parse into a .env file. You've already created the Service Account Token and added OP_SERVICE_ACCOUNT_TOKEN variable to your GitLab CI variables.

  • You use fvm to manage your Flutter SDK versions.

  • You use GitLab (obviously) and have private GitLab repositories in your pubspec.yaml.

  • You've already created an SSH key-pair and added the private key to the consuming repository and the public key to the source repository.

Configuring the Workflow

We're going to start at the beginning, which is configuring the .gitlab-ci.yml workflow rules. In my case, I want the pipeline to trigger on all merge requests and all pushes to the main branches.

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
    - if: $CI_COMMIT_BRANCH == 'dev'
    - if: $CI_COMMIT_BRANCH == 'stage'
    - if: $CI_COMMIT_BRANCH == 'main'

Be sure to modify this according to the environments and branches you have configured for your project. I use three environment, Dev, Stage, and Production. I have my GitLab repositories configured so that all of the branches reflect their environment, except Production which uses main.

Flavors

We use Flutter app Flavors to configure each of our environments and because it's a better name than just ENV. Like I said, setting up the actual app Flavors may not be applicable to your use case, but you'll still want this GitLab CI job to extract the environment.

.setup_flavor:
  stage: flavor
  script:
    - echo "Setting up $FLAVOR flavor..."
    - echo "FLAVOR=$FLAVOR" >> .env
  artifacts:
    reports:
      dotenv: .env
    when: on_success

setup_flavor_dev:
  extends: .setup_flavor
  variables:
    FLAVOR: 'dev'
  rules:
    - if: $CI_COMMIT_REF_NAME == 'dev'
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'dev'

...

To break this down, we're using a "hidden" CI job .setup_flavor to configure the base job that setup_flavor_dev inherits with the extend keyword. This makes the configuration easier to read and maintain.

This job will determine the FLAVOR (environment) variable based on the rules that are set. In this case, we are saying if the commit is referencing dev or if a merge request is targeting dev, then FLAVOR=dev.

Once we know what environment we're in, we persist it in a .env artifact that will be passed to the next job, which will be able to extract the FLAVOR and use it to retrieve secrets from 1Password.

I didn't include the other environments here, so make sure you add those or reference the complete file at the bottom of this guide.

Retrieving Secrets

Next, we're going to retrieve secrets from a 1Password secure note. We're going to use the 1Password CLI v2 or higher in order to use a Service Account Token to retrieve the secrets.

💡
If you haven't already, create the Service Account Token and add OP_SERVICE_ACCOUNT_TOKEN to your GitLab CI variables for this to work.

GitLab has particular rules around the format a.envfile can have, so make sure your secure note does not include any empty lines or comments.

In addition to the GitLab rules, the sed parsing logic used expects the secrets to have single quotes, not double quotes.

For example, the secure note should be formatted like this:

SOME_SECRET = 'some_secret_value'
NEXT_SECRET = 'another_secret_value'

fetch_secrets:
  image: 1password/op:2.29.0
  stage: secrets
  before_script:
    - op user get --me
  script:
    - echo "Retrieving .env secrets from 1Password..."
    - ENVS=$(op item get "myapp - $FLAVOR" --vault "My Secret Vault" --fields label=notesPlain | sed -e 's/\\n/\n/g' -e 's/^"//')
    - echo "$ENVS" > .env
  artifacts:
    reports:
      dotenv: .env
    when: on_success

Using the 1password/op:2.29.0 Docker image, we will have access to the op CLI tools. Next we will fetch the item from the Vault using the filename provided, which includes the FLAVOR. For example, myapp - dev would be the name of the note.

This will parse the plain secure note using sed, then output the values into the .env file, which will again be persisted as an artifact for the next job to pickup. You could opt to extract the artifact creation into another nameless job like we did with the .setup_flavor.

Install Flutter and run those tests

This one is pretty hefty, so let's take it step by step. First, let's setup the base of the job. This will use the Docker image for leoafarias/fvm. In the before script, we extract the Flutter version from the .fvmrc file, install theFlutter SDK, and tell fvm to use that version.

flutter_setup:
  image: leoafarias/fvm
  stage: setup_and_test
  before_script:
    - export FLUTTER_VERSION=$(grep '"flutter":' .fvmrc | sed 's/[^0-9.]//g')
    - fvm install $FLUTTER_VERSION
    - fvm use $FLUTTER_VERSION
  script:
    ...

If you are not using private GitLab packages in your pubspec.yaml, then that's all you need for the before_script. If you are using private packages, then we need to include the SSH key extraction.

💡
You should have already created an SSH key-pair and added the private key to the consuming repository and the public key to the source repository.
flutter_setup:
  ...
  before_script:
    - 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )'
    - eval $(ssh-agent -s)
    - chmod 400 "$SSH_PRIVATE_KEY"
    - ssh-add "$SSH_PRIVATE_KEY"
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    ...

This installs the ssh-agent if not already installed, runs the ssh-agent inside the build environment, stores the SSH key to the agent, and creates the .ssh directory with the right permissions.

In the above, we make the assumption that you've named your GitLab CI file variable as SSH_PRIVATE_KEY. If you named it something different, just update the values appropriately.

Following the official GitLab CI documentation on Using SSH keys with GitLab CI/CD, I was able to get most of the way there. There was one piece missing, which was adding the Self-Hosted GitLab domain to the list of known hosts. You're welcome!

flutter_setup:
  ...
  before_script:
    ...
    - touch ~/.ssh/known_hosts
    - ssh-keyscan -t rsa git.DOMAIN.com >> ~/.ssh/known_hosts
    ...

Now we're getting somewhere! Next, we're going to add the main script, which will run all of the setup commands we need before running the tests. In this case, I want to clean the Flutter cache, get pub dependencies, run build runner, generate localization files, format, analyze, and then test.

flutter_setup:
  ...
  script:
    - fvm flutter clean
    - fvm flutter pub get
    - fvm dart run build_runner build --delete-conflicting-outputs
    - fvm dart run intl_utils:generate
    - fvm dart format .
    - fvm flutter analyze .
    - fvm flutter test

Setting the Stage

We know we have three stages, flavor, secrets, and setup_and_test. So let's add that to the file between workflow and .setup_flavor. Feel free to rename these or add more or less based on your use case.

workflow:
  ...

stages:
  - flavor
  - secrets
  - setup_and_test

.setup_flavor
  ...

Put it altogether now...

So without further ado, let's get to the goods. Here is the complete .gitlab-ci.yml file configuration we put together, with a bonus of FF_TIMESTAMPS which you can remove or disable.

# -----------------------------------------------------------------------------
# This defines the GitLab CI pipeline, which is triggered on all merge requests
# and pushes to the main branches.
# -----------------------------------------------------------------------------

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
    - if: $CI_COMMIT_BRANCH == 'dev'
    - if: $CI_COMMIT_BRANCH == 'stage'
    - if: $CI_COMMIT_BRANCH == 'main'

stages:
  - flavor
  - secrets
  - setup_and_test

# -----------------------------------------------------------------------------
# Extracts the FLAVOR from the current environment and outputs the value into 
# an `.env` file to be used by the next job.
# -----------------------------------------------------------------------------

.setup_flavor:
  stage: flavor
  script:
    - echo "Setting up $FLAVOR flavor..."
    - echo "FLAVOR=$FLAVOR" >> .env
  artifacts:
    reports:
      dotenv: .env
    when: on_success

setup_flavor_dev:
  extends: .setup_flavor
  variables:
    FLAVOR: 'dev'
  rules:
    - if: $CI_COMMIT_REF_NAME == 'dev'
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'dev'

setup_flavor_stage:
  extends: .setup_flavor
  variables:
    FLAVOR: 'stage'
  rules:
    - if: $CI_COMMIT_REF_NAME == 'stage'
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'stage'

setup_flavor_prod:
  extends: .setup_flavor
  variables:
    FLAVOR: 'production'
  rules:
    - if: $CI_COMMIT_REF_NAME == 'main'
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'

# -----------------------------------------------------------------------------
# Uses the 1Password Service Account token to retrieve and parse secrets before
# outputting to an `.env` file, which is required to run the next job.
# -----------------------------------------------------------------------------

fetch_secrets:
  image: 1password/op:2.29.0
  stage: secrets
  before_script:
    # Ensure the 1Password Service Account Token exists.
    - op user get --me
  script:
    # Retrieve and parse secrets from 1Password.
    - echo "Retrieving .env secrets from 1Password..."
    - ENVS=$(op item get "myapp - $FLAVOR" --vault "My Secret Vault" --fields label=notesPlain | sed -e 's/\\n/\n/g' -e 's/^"//')
    - echo "$ENVS" > .env
  artifacts:
    reports:
      dotenv: .env
    when: on_success

# -----------------------------------------------------------------------------
# Sets up FVM and installs Flutter before running build_runner, generating
# localization files, formatting, analyzing, and finally testing.
# -----------------------------------------------------------------------------

flutter_setup:
  image: leoafarias/fvm
  stage: setup_and_test
  before_script:
    # Install ssh-agent if not already installed, it is required by Docker.
    - 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )'
    # Run ssh-agent inside the build environment.
    - eval $(ssh-agent -s)
    # Give the right permissions, otherwise ssh-add will refuse to add files.
    # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store.
    - chmod 400 "$SSH_PRIVATE_KEY"
    - ssh-add "$SSH_PRIVATE_KEY"
    # Create the SSH directory and give it the right permissions.
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    # Add GitLab to the list of known hosts.
    - touch ~/.ssh/known_hosts
    - ssh-keyscan -t rsa git.DOMAIN.com >> ~/.ssh/known_hosts
    # Extract the Flutter version from `.fvmrc`.
    - export FLUTTER_VERSION=$(grep '"flutter":' .fvmrc | sed 's/[^0-9.]//g')
    - fvm install $FLUTTER_VERSION
    - fvm use $FLUTTER_VERSION
  script:
    - fvm flutter clean
    - fvm flutter pub get
    - fvm dart run build_runner build --delete-conflicting-outputs
    - fvm dart run intl_utils:generate
    - fvm dart format .
    - fvm flutter analyze .
    - fvm flutter test
0
Subscribe to my newsletter

Read articles from frankylee kelly directly inside your inbox. Subscribe to the newsletter, and don't miss out.

GitLab-CIsshGitLab SSHci-cd1passwordFlutterDartFVM

Written by

frankylee kelly
frankylee kelly