Orchestrating GitLab CI/CD Workflows Beyond the Docker Paradigm

The architectural decision to utilize or bypass Docker within a GitLab CI/CD pipeline is a pivotal moment in DevOps engineering. While containerization has become the industry standard for ensuring environment parity and isolation, there are numerous technical, security, and resource-based reasons why an organization might choose to execute CI/CD jobs without the standard Docker executor. Understanding the mechanics of GitLab Runner, the nuances of different executors, and the specific configurations required to manage container images and registries—even when not using a Docker-based runner—is essential for any engineer building scalable automation. This deep dive explores the various methods of running GitLab CI/CD, the installation of runners across multiple platforms, the complexities of authentication for container registries, and the specific configurations required when Docker-in-Docker (DinD) or shell-based execution is employed.

The Architecture of GitLab Runner and Executor Selection

At the heart of GitLab CI/CD is the GitLab Runner, a lightweight agent that executes the jobs defined in a .gitlab-ci.yml file. The choice of "executor" determines the environment in which these jobs run. While the Docker executor is highly popular due to its ability to provide clean, isolated environments for every job, it is not the only option available.

When engineers decide to move away from the Docker executor, they typically pivot to the Shell executor or other specialized executors like Kubernetes. The selection of an executor has a direct impact on security, resource overhead, and the ease of managing dependencies.

The Shell Executor Approach

The Shell executor allows GitLab Runner to run jobs directly on the host machine's shell (such as bash, PowerShell, or cmd). In this configuration, the gitlab-runner user is responsible for executing the commands defined in the CI job.

Technical Implementation
The shell executor does not provide the isolation that a container does. This means that if a job requires a specific version of Python or Node.js, that tool must be pre-installed on the host machine where the runner is residing.
Security and Permission Implications
Because the jobs run directly on the host, the gitlab-runner user requires specific permissions to perform necessary tasks. If a job needs to interact with the system, the user must have the appropriate sudo privileges or group permissions, which introduces a larger attack surface compared to the isolated Docker executor.
Capability for Docker Commands
One of the most common reasons to use a Shell executor is to run Docker commands without needing to enable "privileged mode" on a Docker-based runner. By using the Shell executor, the gitlab-runner user can invoke the Docker Engine installed on the host, provided the user is part of the docker group.

Comparison of Runner Execution Methods

The following table outlines the characteristics of the most common execution methods discussed in the context of GitLab CI/CD.

Executor Type	Isolation Level	Dependency Management	Security Risk	Primary Use Case
Docker	High	Handled via Container Image	Low	Standardized, isolated builds
Shell	Low	Handled on Host Machine	High	Running Docker commands or using host tools
Kubernetes	Very High	Handled via Pods	Low	Scaling in orchestrated environments

Installation and Local Execution of GitLab Runner

Before a runner can execute jobs, it must be installed and registered. GitLab provides various installation paths depending on the operating system of the host machine. This flexibility allows developers to run CI/CD jobs locally for testing purposes, simulating the behavior of a remote runner.

Platform-Specific Installation Procedures

The installation of the GitLab Runner binary varies significantly across different operating systems. Engineers must ensure that the appropriate package manager or download method is used to maintain system stability.

macOS Installation
For users on macOS, the simplest method is utilizing the Homebrew package manager.

brew install gitlab-runner

Debian and Ubuntu Installation
For Linux distributions based on Debian, a specialized script is used to add the GitLab repository before installing the package via apt.

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

sudo apt-get install gitlab-runner

CentOS Installation
On CentOS systems, the process involves adding the RPM repository and using the yum package manager.

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh | sudo bash

sudo yum install gitlab-runner

Manual Installation for Linux (AMD64)
For environments where a package manager is not preferred or available, the binary can be downloaded directly from Amazon S3 and placed in the system path.

sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64

Registering a Runner with the Shell Executor

Once the binary is installed, it must be registered with a GitLab instance (such as GitLab.com, GitLab Self-Managed, or GitLab Dedicated). During registration, the user must specify the executor. To use the shell executor as discussed previously, the registration command would be:

sudo gitlab-runner register -n \ --url "https://gitlab.com/" \ --registration-token REGISTRATION_TOKEN \ --executor shell \ --description "My Runner"

Local Job Execution for Testing

A powerful feature of the GitLab Runner is the ability to execute jobs locally without waiting for the remote GitLab server to pick them up. This is invaluable for debugging .gitlab-ci.yml syntax or script logic.

Executing with Docker
If the runner is configured to use the Docker executor, jobs can be run locally using the exec command.

gitlab-runner exec docker <name_of_the_job_in_gitlab-ci.yml>

Example:
gitlab-runner exec docker localbuild

Executing with Shell
If the developer wishes to test how the job behaves in a shell environment, they use the shell flag.

gitlab-runner exec shell <name_of_the_job_in_gitlab-ci.yml>

Example:
gitlab-runner exec shell localbuild

Container Registry Authentication and Credential Management

Even when not using the Docker executor to run the CI job itself, GitLab CI/CD often involves interacting with container registries to push or pull images. Managing authentication for these registries is a complex task involving credential helpers, environment variables, and configuration files.

The GitLab Container Registry and CIJOBTOKEN

When using the GitLab Container Registry hosted on the same instance as the project, GitLab simplifies authentication by providing default credentials.

Authentication Mechanism
The CI_JOB_TOKEN is used for authentication. This token is a short-lived credential that allows the job to interact with the registry.
Permission Requirements
To use the job token for a private image, the user initiating the job must hold specific roles:
Developer
Maintainer
Owner
Cross-Project Access
By default, access to private images in other projects is disabled. For a job in Project A to pull an image from Project B using the CI_JOB_TOKEN, Project B must be configured to allow authentication from Project A.

Advanced Credential Management via Credential Helpers

For more complex scenarios, such as interacting with Amazon Elastic Container Registry (ECR), standard tokens are insufficient. In these cases, "Credential Helpers" are used to manage authentication dynamically.

The Requirement for $PATH
To use a credential helper, the specific binary (e.g., docker-credential-ecr-login) must be available in the GitLab Runner's $PATH. If the binary is missing, the runner will fail to authenticate with the registry.
Configuring AWS ECR Access
To facilitate access to a private ECR registry, an engineer can use one of two methods to make the GitLab Runner aware of the helper.
Method 1: CI/CD Variable
A variable named DOCKER_AUTH_CONFIG can be created in the GitLab UI. The value of this variable should be a JSON object that points to the helper.

{ "credHelpers": { "<aws_account_id>.dkr.ecr.<region>.amazonaws.com": "ecr-login" } }

Method 2: Local Configuration (Self-Managed)
For self-managed runners, the JSON configuration can be added directly to the runner's local filesystem:
${GITLAB_RUNNER_HOME}/.docker/config.json
Using the credsStore
An alternative to specific helpers is the use of the credsStore property. This can be applied globally or to specific registries.
Global ECR Credential Store:
{ "credsStore": "ecr-login" }
Note: If credsStore is used for all registries, pulling from public registries like Docker Hub may fail because the Docker daemon will attempt to use the ECR credentials for all requests.
Specific Registry Credential Store:
{ "credsStore": "osxkeychain" }
Region Configuration for AWS
When using credsStore with ecr-login, the AWS region must be explicitly set in the AWS shared configuration file located at ~/.aws/config. The ECR Credential Helper requires this region to retrieve the necessary authorization tokens.

Priority of Configuration Reading

The GitLab Runner follows a strict hierarchy when determining which configuration to use for Docker authentication. Understanding this order is critical for troubleshooting authentication failures.

The config.json file located in the /root/.docker directory.
The DOCKER_AUTH_CONFIG CI/CD variable.
The DOCKER_AUTH_CONFIG environment variable defined in the runner's config.toml.
The config.json file in the $HOME/.docker directory of the user running the process. If the --user flag is used to run the process as an unprivileged user, the home directory of the main runner process user is utilized.

Docker-in-Docker (DinD) and Kubernetes Orchestration

In advanced CI/CD workflows, particularly those running on Kubernetes, the concept of Docker-in-Docker (DinD) is often employed to allow a containerized job to build other container images. This requires specific, and often sensitive, configurations.

Secure DinD in Kubernetes

Running Docker inside a Kubernetes-managed pod requires the pod to be "privileged." Without this, the Docker daemon inside the container cannot function correctly.

The Standard TLS Configuration
In a standard setup, the DOCKER_HOST is pointed to a service container, and TLS is used to secure the communication.

DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_TLS_VERIFY: 1
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"

Disabling TLS in Kubernetes
In some Kubernetes environments, TLS might be disabled to simplify the setup or due to specific architectural constraints. To disable TLS, the configuration must be modified to remove the empty directory volumes and change the port.
Required Modifications for No-TLS:
Remove the [[runners.kubernetes.volumes.empty_dir]] section from the values.yml file.
Change the port from 2376 to 2375.
Set DOCKER_HOST: tcp://docker:2375.
Set DOCKER_TLS_CERTDIR: "" to instruct Docker to start without TLS.

Example Kubernetes Runner Configuration:

yaml runners: tags: "no-tls-dind-kubernetes-runner" config: | [[runners]] [runners.kubernetes] image = "ubuntu:20.04" privileged = true

Analysis of Operational Strategies

The decision to implement GitLab CI/CD without the standard Docker executor involves a complex trade-off between security, isolation, and administrative overhead. Utilizing the Shell executor offers a path of least resistance for teams already managing robust build servers, as it allows for direct access to local tools and existing Docker installations. However, this approach necessitates a rigorous approach to host security and dependency management, as the boundary between the CI job and the host operating system is significantly more porous than in a containerized environment.

For organizations operating at scale, particularly those utilizing Kubernetes, the complexity shifts toward managing the lifecycle of "privileged" containers. The ability to run Docker-in-Docker (DinD) provides the necessary capability to build images within a containerized workflow, but it introduces specific requirements for TLS configuration and port management (shifting from 2376 to 2375) to accommodate non-TLS environments.

Furthermore, the management of container registries remains a sophisticated component of the pipeline, regardless of the executor used. The hierarchy of credential lookup—ranging from the root-level config.json to CI/CD variables—requires precise orchestration to ensure that private images in ECR or the GitLab Container Registry are accessible without compromising the security of the entire runner fleet. The use of Credential Helpers is an essential mechanism for integrating with cloud-native registries, provided that the necessary binaries are correctly placed within the runner's $PATH. Ultimately, a successful GitLab CI/CD implementation requires a deep understanding of how the runner interacts with both the host system and the external registry ecosystem.