Here is a clarification on how GitLab authenticates to HashiCorp Vault and how to set up a CI/CD pipeline to access secrets.

You can find this information in the official Vault and GitLab guides. Since these details are scattered across various guides and references, I created this overview.

Here you will find links to the original guides as well.

TL;TR

GitLab creates JWT tokens (ID Tokens) to access Vault. Those tokens are signed with a private key that Vault can verify to allow the access.
In the GitLab CI Job, you also need to specify a Vault Role.
In the Role you configure information to limit the access from specific GitLab subjects (users, namespaces, projects, branches ...).
That information is passed in the form of Token Claims included by Gitlab.
If token claims match the ones defined in the requested role, Vault allow the access.
The job can only access the Vault paths allowed by the Policies specified in the Role.

ID Tokens

To integrate with Hashicorp Vault, Gitlab uses JSON Web Tokens (JWTs) that can be added to a GitLab CI/CD job. Those are called ID Tokens.

ID Tokens authentication overview

Here is an example of the flow when GitLab CI uses a JWT to authenticate with HashiCorp Vault and retrieve secrets.

Note that in this flow, Vault is configured to use the JWT auth method, which allows it to verify JWT tokens directly without relying on an OIDC provider. The JWT auth method is configured with the expected issuer, audience, and other settings to verify the JWT tokens.

Authentication flow:

In your Vault, you need to configure the JWT Authentication method. See details in GitLab doc Authenticating and reading secrets with HashiCorp Vault (search for "configure the JWT Authentication method").
Add roles and policies to enable specific GitLab projects to access limited vault paths (see below for details about how to use GitLab JWT claims).
GitLab CI generates a JWT token that contains the required claims, such as the issuer, subject, and audience. The token is signed using a private key.
GitLab CI sends it to Vault as a Bearer token in the Authorization header.
HashiCorp Vault verifies the JWT by checking the signature, issuer, and audience. Here, the discovery_url is not directly used in the authentication flow. However, it is still used to retrieve the JSON Web Key Set (JWKS) document, which contains the public keys used to verify the JWT signature.
If the JWT is valid, Vault extracts the claims and uses them to authorize the request.
If the authentication is successful, Vault returns an authentication token.
Runner reads secrets from the HashiCorp Vault.

Enabling the JWT authentication method on Vault

The JWT auth method is part of the more general JWT/OIDC authentication provided by Vault. To enable it:

vault auth enable jwt

To configure it for GitLab CI integration:

$ vault write auth/jwt/config \
    oidc_discovery_url="https://gitlab.example.com" \
    bound_issuer="https://gitlab.example.com"

Vault doc - JWT/OIDC auth method - JWT Configuration
GitLab doc - Authenticating and reading secrets with HashiCorp Vault (search for "configure the JWT Authentication method").

Restricting the access with Tokens claims and Policies

In a JWT, a claim appears as a name/value pair, where the name is always a string and the value can be any JSON value. In this context, we refer to the name of the key.

Eg: Gitlab JWT token payload

{
  "namespace_id": "72",
  "namespace_path": "my-group",
  "project_id": "20",
  "project_path": "my-group/my-project",
  "user_id": "1",
  "user_login": "sample-user",
  "user_email": "sample-user@example.com",
  "user_identities": [
      {"provider": "github", "extern_uid": "2435223452345"},
      {"provider": "bitbucket", "extern_uid": "john.smith"},
  ],
  "pipeline_id": "574",
  "pipeline_source": "push",
  "job_id": "302",
  "ref": "feature-branch-1",
  "ref_type": "branch",
  "ref_path": "refs/heads/feature-branch-1",
  "ref_protected": "false",
  "groups_direct": ["mygroup/mysubgroup", "myothergroup/myothersubgroup"],
  "environment": "test-environment2",
  "environment_protected": "false",
  "deployment_tier": "testing",
  "environment_action": "start",
  "runner_id": 1,
  "runner_environment": "self-hosted",
  "sha": "714a629c0b401fdce83e847fc9589983fc6f46bc",
  "project_visibility": "public",
  "ci_config_ref_uri": "gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main",
  "ci_config_sha": "714a629c0b401fdce83e847fc9589983fc6f46bc",
  "jti": "235b3a54-b797-45c7-ae9a-f72d7bc6ef5b",
  "iss": "https://gitlab.example.com",
  "iat": 1681395193,
  "nbf": 1681395188,
  "exp": 1681398793,
  "sub": "project_path:my-group/my-project:ref_type:branch:ref:feature-branch-1",
  "aud": "https://vault.example.com"
}

As you can see, there are many custom claims related to GitLab.

Claims are used to gather information about the subject requesting the authentication. If there is a mismatch, the authentication fails.

Using Roles to define Bound Claims

💡

The default endpoint for JWT roles is "auth/jwt/role/" (you could also have additional JWT paths like "auth/<path-to-jwt-backend>/role").

When configuring roles in Vault, you can use bound claims to match against the JWT claims and restrict which secrets each CI/CD job has access to.

The authentication request from GitLab is linked to a specific role declared in the pipeline configuration.

So:

You need to create a Vault role. In that role, you can use any of the claims that GitLab includes in its JWT token.
When you configure a GitLab job to connect to Vault you will declare a role that will be used to authorize the access to the requested secrets.
During the authentication process, Vault will compare those claims with the ones specified in the role.

This is an example of role limiting the access from a particular project and branch:

Vault Role

$ $ vault write auth/jwt/role/myproject-staging - <<EOF
{
  "role_type": "jwt",
  "policies": ["myproject-staging"],
  "token_explicit_max_ttl": 60,
  "user_claim": "user_email",
  "bound_audiences": "https://vault.example.com",
  "bound_claims": {
    "project_id": "22",
    "ref": "master",
    "ref_type": "branch"
  }
}
EOF

Using Vault Policies to limit access to secrets

💡

The endpoint for Policies is "sys/policies/acl". You can list and operate on policies with the "vault policy" command.

As you can see in the example above, in the same role you can specify a list of policies that will limit the access to secrets.

Example of Policy:

vault policy write myproject-production - <<EOF
# Read-only permission on 'ops/data/production/*' path

path "ops/data/production/*" {
  capabilities = [ "read" ]
}
EOF

Vault doc - Policies
- Policies provide a declarative way to grant or forbid access to certain paths and operations in Vault.
GitLab doc - Authenticating and reading secrets with HashiCorp Vault
- Example

Integrating Gitlab and Hashicorp Vault: A Complete Overview

Table of contents