The architecture of a GitLab CI/CD pipeline relies heavily on the precise orchestration of job execution, which is governed by configuration files, most notably the .gitlab-ci.yml file. At the heart of this orchestration lies the ability to control the lifecycle of a job—determining exactly when a task should run, which branches it should target, and which events should trigger its execution. For years, the primary mechanism for this control was the only keyword. However, as GitLab has matured from a simple repository hosting service into a complex DevOps platform, the way developers manage pipeline triggers has undergone a significant paradigm shift. This evolution has moved from the relatively straightforward but often brittle only and agit keywords toward the more robust and powerful rules syntax. Understanding the nuances of how GitLab evaluates job inclusion, the pitfalls of legacy configurations, and the modern best practices for pipeline logic is essential for any engineer attempting to maintain a scalable, non-redundant, and efficient CI/CD ecosystem.
Fundamental Concepts of GitLab CI/CD Jobs
Before addressing the specific logic of the only keyword, one must understand the fundamental nature of a job within the GitLab ecosystem. A job is the atomic unit of a pipeline. It represents a specific set of instructions that must be carried out to achieve a goal, such as compiling source code, running a suite of unit tests, or deploying an application to a production environment.
A job is characterized by several core attributes:
- Execution Environment: Every job executes on a GitLab Runner, which is often an instantiated Docker container. This allows for an isolated, reproducible environment where dependencies are strictly controlled.
- Independence: Jobs are designed to run independently from one another. While they may exist within the same pipeline, the failure of one job does not inherently halt the execution of others unless specific dependencies, such as the
needskeyword, are configured. - Execution Logs: Every job generates a comprehensive log. This log is the primary diagnostic tool for engineers, providing a full execution history that is vital for troubleshooting failures in the build or deployment process.
- Stages and Parallelism: Jobs are organized into stages (e.g.,
build,test,deploy). Within a single stage, all defined jobs can run in parallel, significantly reducing the total "wall clock" time of the pipeline. Stages themselves execute in a strict, sequential order. - Artifacts and Caching: Jobs can produce artifacts, which are small, essential files that are guaranteed to be passed to subsequent jobs in the same pipeline. Conversely, jobs use cache to store large amounts of data (like downloaded dependencies) that are not guaranteed to be present in future runs but are optimized for speed.
The configuration of these jobs is handled through YAML keywords that define everything from the Docker image used to the specific shell commands to be executed.
The Mechanics and Syntax of the Only Keyword
The only keyword is a legacy configuration tool used to restrict job execution to specific Git references or pipeline triggers. When a developer uses only, they are explicitly defining a "whitelist" of conditions that must be met for the job to be added to the pipeline.
The syntax of only can be applied to various Git references, including:
- Branches: Restricting a job to run only when changes are pushed to a specific branch, such as
main. - Tags: Rest-ricting a job to run only when a Git tag is created, which is a common practice for triggering release pipelines.
- Merge Requests: Ensuring a job runs specifically within the context of a merge request event.
- Regular Expressions: Using complex patterns to match branch names, such as
/^issue-.*$/. - Web Triggers: Allowing a job to run when a user manually initiates a pipeline via the GitLab UI.
The behavior of only is subject to specific default settings that can lead to confusion. If a job does not utilize only, except, or rules, GitLab defaults the job's execution scope to both branches and tags. This means the job will run on every branch push and every tag creation by default.
An important technical detail regarding the shorthand notation is that using only or except without additional keywords is functionally equivalent to targeting refs. For instance, the following two configurations result in identical behavior:
```yaml
job1:
script: echo
only:
- branches
job2:
script: echo
only:
refs:
- branches
```
This level of equivalence demonstrates that only is fundamentally a filter applied to the Git reference (refs) of the pipeline.
Deprecation and the Transition to Rules
While only and except were the standard for much of GitLab's history, they are now considered deprecated in favor of the rules keyword. The primary reason for this deprecation is the complexity of modern DevOps workflows. The rules keyword provides a more granular and logical way to define job execution based on variable states, pipeline sources, and complex conditional logic.
Specifically, the following usages of only and except are officially deprecated:
only:variables: Controlling job execution based on the status of CI/CD variables.except:variables: Preventing job execution based on the status of CI/CD variables.
To manage job creation based on variables or complex regular expressions, engineers should transition to using rules:if.
The transition is not merely a matter of syntax but a matter of preventing "double pipelines." A common anti-pattern in GitLab CI is the creation of duplicate pipelines—one for a branch push and one for a merge request. This occurs when a job uses rules in a way that matches both push and event sources without a global workflow:rules configuration to arbitrate.
Example of a problematic configuration:
yaml
job:
script: echo "This job creates double pipelines!"
rules:
- if: $CI_PIPELINE_SOURCE == "push"
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
In the scenario above, a single push to a branch that has an open merge request will trigger two separate pipelines. To prevent this, developers must implement workflow:rules at the top level of the .gitlab-ci.yml file.
The Danger of Mixing Only/Except with Rules
One of the most significant risks in GitLab CI configuration is the simultaneous use of only/except and rules within the same pipeline. While this might not trigger a YAML syntax error, it introduces "silent" logical errors that are notoriously difficult to debug.
The underlying issue is that only/implements and rules have different default behaviors. For example:
- A job with no rules defaults to
except: merge_requests. This means it will run on all branch pipelines but will specifically be excluded from merge request pipelines. - A job configured with
rulesmight be explicitly set to run only onmerge_request_event.
When these two types of jobs exist in the same pipeline, the developer may see a single job running in a branch pipeline and a completely different set of jobs running in a merge request pipeline, leading to a fragmented and unpredictable CI process.
To illustrate, consider this configuration:
```yaml
job-with-no-rules:
script: echo "This job runs in branch pipelines."
job-with-rules:
script: echo "This job runs in merge request pipelines."
rules:
- if: $CIPIPELINESOURCE == "mergerequestevent"
```
In this instance, for every change pushed to a branch with an open merge request, two pipelines are generated. One pipeline (the branch pipeline) executes job-with-no-rules, and the other (the merge request pipeline) executes job-with-rules.
Advanced Configuration: Hidden Jobs, Templates, and Inheritance
To maintain clean and DRY (Don't Repeat Yourself) configurations, GitLab provides several mechanisms for job abstraction.
Hidden Jobs and Templates
A job name starting with a dot (e.g., .template_job) is considered a "hidden job." Hidden jobs are not processed as actual tasks by the GitLab Runner; instead, they serve as templates for other jobs. They do not require a script keyword but must contain valid YAML.
yaml
.hidden_job:
script:
- run test
The Evolution from Anchors to Extends and Reference Tags
Historically, developers used YAML anchors (& and *) to reuse code. However, this is now considered a sub-optimal practice because it can lead to hard-to-read and over-engineered configurations.
A sub-optimal approach using anchors:
```yaml
.defaultscripts: &defaultscripts
- ./default-script1.sh
- ./default-script2.sh
.jobtemplate: &jobconfiguration
image: ruby:2.6
services:
- postgres
- redis
job1:
<<: *jobconfiguration
script:
- *defaultscripts
- ./job-script.sh
```
The modern, recommended approach uses extends and the !reference tag. The extends keyword allows a job to inherit configuration from a template, while !reference allows for the surgical insertion of specific parts of another job's configuration (like a specific list of scripts).
The optimized approach:
```yaml
.default_scripts:
script:
- ./default-script1.sh
- ./default-script2.sh
.job_template:
image: ruby:2.6
services:
- postgres
- redis
job1:
extends: .jobtemplate
script:
- !reference [.defaultscripts, script]
- ./job-script.sh
```
Global Defaults and Controlled Inheritance
The default keyword allows for the establishment of global parameters, such as a standard Docker image or a common before_script command. This ensures consistency across all jobs in the pipeline.
```yaml
default:
image: 'ruby:2.4'
before_script:
- echo Hello World
rspec-job:
script: bundle exec rspec
```
However, complex pipelines may require certain jobs to deviate from these defaults. The inherit keyword provides granular control over what a job inherits from the default block:
inherit: [image]allows a job to use the global image but ignore the globalbefore_script.inherit: variables: falseallows a job to strip away global CI/CD variables to ensure a clean environment.
Troubleshooting Included Files and Stage Mismatches
A common point of failure when modularizing .gitlab-ci.yml files through the include keyword is the "missing job" phenomenon. Developers often find that while their main file includes a local file, only the first job from the included file appears in the pipeline.
This is rarely a bug in the include mechanism itself, but rather a logical error in stage configuration. If an included job is assigned to a stage that is not explicitly defined in the parent (outer) .gitlab-ci.yml file, that job will fail to appear in the parsed pipeline.
Consider this failure scenario:
Contents of gitlab-ci.yml:
```yaml
stages:
- build
- test
- deploy
include:
- local: 'ci/build.gitlab-ci.yml'
```
Contents of ci/build.gitlab-ci.yml:
```yaml
build:
script: echo "Build"
stage: build
another-build:
script: echo "Another build"
stage: custom-stage # This stage is not defined in the main file
```
In the merged view, the another-build job will be absent because the stages list in the main file does not recognize custom-stage. The resulting stages list will default to:
- .pre
- build
- test
- deploy
- .post
To resolve this, every stage used in any included file must be present in the top-level stages declaration.
Analysis of Pipeline Scalability and Best Practices
The transition from only to rules and from YAML anchors to extends represents a broader movement toward "Configuration as Code" maturity. For an engineer to successfully manage large-scale GitLab CI/CD environments, they must move away from the reactive debugging of legacy keywords and toward a proactive, rule-based architecture.
The key takeaways for maintaining pipeline integrity are:
- Prioritize
rulesoveronly/exceptto avoid the complexities of implicit branch/tag behavior and to enable variable-based logic. - Implement
workflow:rulesto strictly control the creation of pipelines, particularly when dealing with both branch and merge request triggers, to prevent resource-wasting duplicate pipelines. - Utilize
extendsand!referencefor code reuse to maintain readability and prevent the "spaghetti YAML" often caused by heavy anchor usage. - Ensure all stages used in modularized (included) configuration files are explicitly declared in the primary
.gitlab-ci.ymlfile to prevent job loss during the merging process. - Adopt versioned, public Docker images for job execution to ensure that the environment remains consistent and that the pipeline is not susceptible to breaking changes in underlying OS packages.
By adhering to these architectural principles, developers can transform the CI/CD pipeline from a fragile collection of scripts into a robust, scalable, and highly observable engine for continuous delivery.