The Mechanics and Evolution of CI_JOB_JWT_V2 in GitLab CI/CD Pipelines

The architectural foundation of secure automation within modern DevOps relies heavily on the ability of a CI/CD pipeline to prove its identity to external services without relying on long-lived, static credentials. In the GitLab ecosystem, this identity verification has historically been centered around the CI_JOB_JWT_V2 variable. This predefined variable serves as a JSON Web Token (JWT) that allows a GitLab Runner to authenticate with third-party systems, such as AWS, HashiCorp Vault, or Akeyless, by leveraging the OpenID Connect (OIDC) protocol. By providing a signed, time-bound assertion of the job's identity, CI_JOB_JWT_V2 eliminates the need to store "secret" access keys within GitLab project variables, which are often targets for leakage or compromise.

The fundamental utility of CI_JOB_JWT_V2 is its nature as an RS256 signed token. When a pipeline job is initiated, GitLab generates a unique token specifically for that execution. This token is not static; it is ephemeral, meaning it is valid only for the duration of the pipeline job's lifecycle. Once the job finishes, the token expires and becomes useless for any further authentication attempts. This temporal limitation is a critical security feature that mitigates the risk of token theft; even if a token is intercepted, its window of utility is extremely narrow.

As the ecosystem has evolved, particularly leading into GitLab 17 and following the introductions in version 15.7 and 15.9, the industry has shifted toward more granular "ID tokens." While CI_JOB_JWT_V2 provided a standardized way to pass identity, it suffered from a static audience claim (aud:), which limited the flexibility of how external providers could validate the token. The transition to ID tokens allows developers to define the audience explicitly, ensuring that a token intended for AWS cannot be maliciously repurposed for another service, thereby tightening the security perimeter of the entire software delivery lifecycle.

Technical Specifications and Token Properties

The CI_JOB_JWT_V2 variable is a predefined environment variable injected into the GitLab CI/CD job environment. Its primary purpose is to facilitate OIDC (OpenID Connect) flows.

The impact of using an RS256 signed token means that the receiving party (the Identity Provider or IdP) can verify the token's authenticity using GitLab's public keys. This creates a trust relationship where the cloud provider trusts GitLab to vouch for the identity of the job. If a job is compromised, the blast radius is limited because the token is short-lived and tied to a specific project and pipeline ID.

Integration with AWS via OpenID Connect (OIDC)

One of the most common use cases for CI_JOB_JWT_V2 is establishing a trust relationship with Amazon Web Services (AWS). This process allows a GitLab pipeline to assume an IAM role dynamically.

The OIDC Workflow for AWS

The interaction between GitLab and AWS follows a precise sequence of handshakes to ensure that only authorized pipelines can modify infrastructure.

1. The developer pushes infrastructure code (e.g., Terraform or AWS CDK) to a GitLab repository.
2. The GitLab Runner executes the job and generates the CI_JOB_JWT_V2 token.
3. The Runner invokes the AWS Security Token Service (STS) using the AssumeRoleWithWebIdentity API call.
4. The CI_JOB_JWT_V2 token is passed as the --web-identity-token parameter.
5. AWS validates the token against the configured GitLab OIDC Provider URL (https://gitlab.com).
6. Upon successful validation, AWS issues Temporary Security Credentials (Access Key ID, Secret Access Key, and Session Token).
7. The Runner uses these temporary credentials to perform actions, such as deploying an S3 bucket.
8. Once the job terminates, the temporary credentials expire.

Implementing the AssumeRole Command

To execute this transition from a JWT to AWS credentials, the following command sequence is typically utilized within the .gitlab-ci.yml script section:

bash STS=$(aws sts assume-role-with-web-identity --role-arn ${ROLE_ARN} --role-session-name "gitlab-${CI_PROJECT_ID}-${CI_PIPELINE_ID}" --web-identity-token ${CI_JOB_JWT_V2} --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' --output text) export AWS_ACCESS_KEY_ID="${STS[0]}" export AWS_SECRET_ACCESS_KEY="${STS[1]}" export AWS_SESSION_TOKEN="${STS[2]}" aws sts get-caller-identity

In this implementation, the ROLE_ARN must be stored as a project variable. The use of CI_PROJECT_ID and CI_PIPELINE_ID in the session name ensures that each session is uniquely identifiable in AWS CloudTrail logs, providing high-resolution auditability. The AWS_SESSION_TOKEN is mandatory for dynamic credentials; omitting it will result in authentication failure.

Managing Secrets with Akeyless and HashiCorp Vault

Beyond cloud infrastructure, CI_JOB_JWT_V2 is used to fetch application-level secrets from specialized secret management vaults, ensuring that production API keys are never stored in plain text within GitLab.

Akeyless Integration

The Akeyless plugin for GitLab uses the OAuth 2.0 / JWT Authentication Method. This allows the pipeline to authenticate as a trusted entity and retrieve secrets (such as MySQL database credentials) on the fly. To configure this, an administrator must create a GitLab Auth Method in the Akeyless console under "Users & Auth Methods" > "New" > "OAuth 2.0 / JWT".

HashiCorp Vault Implementation

In a Vault scenario, the CI_JOB_JWT_V2 token (or the newer ID tokens) is used to perform a login against the Vault JWT auth backend. This process removes the need for the "secret-zero" problem, where one would otherwise need a secret to get a secret.

The following configuration demonstrates how to request a specific ID token and use it to authenticate with Vault:

yaml deploy_to_production: stage: deploy id_tokens: VAULT_JWT: aud: https://vault.example.com rules: - if: $CI_COMMIT_BRANCH == "main" script: - | export VAULT_CLIENT_TOKEN=$(curl --request POST --data "{"jwt":"$VAULT_JWT","role":"my-project-role"}" $VAULT_ADDR/v1/auth/jwt/login | jq -r .auth.client_token) - export PRODUCTION_API_KEY=$(curl -H "X-Vault-Token: $VAULT_CLIENT_TOKEN" $VAULT_ADDR/v1/secret/data/my-app/production | jq -r .data.data.api_key) - ./deploy.sh --api-key "$PRODUCTION_API_KEY"

By utilizing the aud (audience) claim in the id_tokens block, the token is explicitly targeted at https://vault.example.com, preventing it from being used at other endpoints. This architecture ensures that the PRODUCTION_API_KEY only exists in the memory of the running job and is never written to the GitLab database.

Transition to ID Tokens and GitLab 17 Security

Starting with GitLab 15.7 and reinforced in version 15.9 and beyond, GitLab has deprecated the predefined CI_JOB_JWT, CI_JOB_JWT_V1, and CI_JOB_JWT_V2 variables in favor of customizable ID tokens.

Differences Between CIJOBJWT_V2 and ID Tokens

The shift toward ID tokens addresses several security and operational gaps:

• Audience Customization: CI_JOB_JWT_V2 had a static audience claim. ID tokens allow the user to specify a single string, an array of strings, or a CI/CD variable as the audience. This allows the same pipeline to interact with multiple different providers (e.g., AWS and GCP) using tokens that are uniquely identified for each provider.
• Granular Access Control: Previously, CI_JOB_JWT_V2 was available across the entire pipeline. ID tokens can be restricted to specific jobs. This significantly reduces the risk of a compromised job in an early stage (like a linting job) leaking a token that could be used to assume a high-privilege production role.
• OIDC Compliance: ID tokens are designed specifically to support OpenID Connect more robustly, aligning GitLab with industry standards for identity federation.

Troubleshooting and Operational Challenges

Despite the robustness of the OIDC flow, certain failure modes exist when using CI_JOB_JWT_V2 at scale.

The InvalidIdentityToken Error in Parallel Jobs

A known issue occurs when multiple jobs attempt to call AssumeRoleWithWebIdentity in AWS IAM simultaneously. This manifests as the InvalidIdentityToken error:

An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: Couldn't retrieve verification key from your identity provider

This error is typically not a result of an incorrect token, but rather a transient failure in AWS's ability to retrieve the verification keys from the GitLab OIDC provider during high-concurrency bursts. The current workaround is to implement a retry mechanism within the CI script. If the first attempt to assume the role fails, the job should wait and retry the call.

AWS CDK Implementation Logic

For those using the AWS Cloud Development Kit (CDK) v2, the OIDC provider and role must be defined programmatically. The following TypeScript logic illustrates the creation of the trust relationship:

```typescript
const oidcProvider = new iam.OpenIdConnectProvider(this, 'Provider', {
url: props.gitLabURI,
clientIds: [ props.gitLabURI ],
});

const role = new iam.Role(this, 'Role', {
assumedBy: new iam.OpenIdConnectPrincipal(oidcProvider).withConditions({
'StringEquals': {
[ props.gitLabURI.substring(8) + ':sub' ]: 'project_path:' + props.gitLabProjectPath,
}
}),
description: 'GitLab pipeline role',
});
```

The withConditions block is the most critical part of this configuration. It ensures that only tokens coming from a specific GitLab project path can assume the role. Without this condition, any project on GitLab could potentially assume the role if they knew the ARN.

Comprehensive Comparison of Authentication Methods

The evolution of secret management in GitLab shows a clear trajectory from static to dynamic identity.

Governance and Strategic Implementation

The technical deployment of CI_JOB_JWT_V2 and ID tokens must be supported by strong governance practices to be effective. A resilient development lifecycle requires more than just a token; it requires a policy-driven approach to identity.

• Secret Rotation Policies: While OIDC removes the need for long-lived AWS keys, the secrets retrieved from Vault or Akeyless still need a defined lifespan. Implementing a rotation policy ensures that even if a secret is retrieved and leaked from a job's memory, it will soon become invalid.
• Least Privilege Access: The IAM roles assumed via CI_JOB_JWT_V2 should be scoped to the minimum necessary permissions. For example, a role used for cdk deploy should only have permissions for the specific resources it manages, rather than AdministratorAccess.
• Audit Logging: By using unique session names (combining CI_PROJECT_ID and CI_PIPELINE_ID), security teams can trace every single AWS API call back to a specific pipeline execution in GitLab.

Conclusion

The CI_JOB_JWT_V2 variable represents a pivotal shift in how CI/CD pipelines handle identity, moving away from the dangerous practice of storing long-lived credentials in configuration files. By leveraging the RS256 signed JWT, GitLab provides a secure mechanism for runners to authenticate with external providers via OIDC. While CI_JOB_JWT_V2 provided the initial breakthrough, the transition to ID tokens in GitLab 15.7 and 17 offers the necessary granularity—specifically through customizable audience claims and job-level restrictions—to meet the security requirements of enterprise-scale environments. The integration with AWS, Akeyless, and HashiCorp Vault demonstrates a mature ecosystem where the "identity" of a job is used as a temporary key to unlock specific, time-bound permissions, drastically reducing the attack surface of the software supply chain.

Sources

1. Akeyless Tutorials
2. Ventx Blog
3. Xebia Blog
4. GitLab Issues
5. GoRegulus