GitLab Runner Docker Implementation and Orchestration

Published by TechMarmot Editorail Staff on | Category GitLab CI

The deployment of GitLab Runner within a Dockerized environment represents a sophisticated intersection of continuous integration and containerization. By leveraging Docker as the primary mechanism for executing CI/CD pipelines, organizations can achieve a level of environment parity and isolation that is unattainable with shell-based executors. The GitLab Runner Docker image is specifically engineered to encapsulate all necessary dependencies required to operate the runner agent and execute complex CI/CD jobs within isolated containers. These images typically utilize Ubuntu or Alpine Linux as their foundational base, effectively wrapping the standard gitlab-runner command. This architecture essentially delegates full control over the Docker daemon to each single Runner container, ensuring that the lifecycle of a build job is strictly tied to the lifecycle of its container.

Dockerized Runner Architecture and Execution

The fundamental nature of running GitLab Runner inside a Docker container is that it mirrors the behavior of a direct host installation but adds a layer of abstraction. Every command executed via the runner is fundamentally an equivalent of a docker run operation. For instance, when a user invokes the runner command gitlab-runner <runner command and options...>, the underlying system is executing docker run <chosen docker options...> gitlab/gitlab-runner <runner command and options...>.

A practical application of this is the retrieval of top-level help information. Instead of installing the binary on a local machine, a user can execute the following command to view the available options:

docker run --rm -t -i gitlab/gitlab-runner --help

The output of this command reveals the identity of the tool: gitlab-runner - a GitLab Runner. The usage pattern follows a specific hierarchy: gitlab-runner [global options] command [command options] [arguments...]. As of version 18.10.1 (3b43bf9f), the software maintains a strict compatibility matrix. An essential characteristic of the GitLab Runner images is their flexibility regarding the Docker Engine version; the versions of the Docker Engine and the GitLab Runner container image do not need to be identical, as the images are designed to be both backwards and forwards compatible.

Registration and Configuration of the Docker Executor

To utilize Docker for CI/CD job execution, a runner must be registered specifically with the Docker executor. This process involves defining the environment in which the jobs will run and ensuring that the necessary services are available. A critical component of this setup is the use of templates to supply services. For example, a user might create a temporary template file at /tmp/test-config.template.toml with the following content:

toml [[runners]] [runners.docker] [[runners.docker.services]] name = "postgres:latest" [[runners.docker.services]] name = "mysql:latest"

Once this template is defined, the runner can be registered using a specific command sequence:

sudo gitlab-runner register --url "https://gitlab.example.com/" --token "$RUNNER_TOKEN" --description "docker-ruby:2.6" --executor "docker" --template-config /tmp/test-config.template.toml --docker-image ruby:3.3

In this specific execution, the registered runner utilizes the ruby:3.3 Docker image as the primary environment. Simultaneously, it initiates two auxiliary services: postgres:latest and mysql:latest. Both of these services are accessible throughout the duration of the build process, allowing the primary job container to interact with the database services for integration testing.

Image Specifications and Requirements

The image keyword in a .gitlab-ci.yml file specifies the Docker image the Docker executor must use to execute the CI/CD jobs. By default, the executor is configured to pull images from Docker Hub. However, administrators can override this behavior by modifying the gitlab-runner/config.toml file to specify a different registry location or to enforce a pull policy that prioritizes local images.

For an image to be compatible with a GitLab CI/CD job, it must meet a set of minimum software requirements. Specifically, the image must have the following applications installed:

  • sh
  • or bash
  • grep

If these utilities are missing, the executor will fail to orchestrate the job scripts, as these tools are essential for the internal shell commands GitLab uses to manage the build lifecycle.

Advanced Networking and Service Communication

The communication between the main build container and its associated services depends heavily on the network mode configured in the config.toml file. By default, the system uses the bridge network mode, which is the standard if the FF_NETWORK_PER_BUILD feature is disabled.

The available networking modes include:

  • bridge: The default mode, utilizing the Docker bridge network.
  • host: This mode allows the container to use the host's network stack directly.
  • none: Disables networking entirely, which is generally not recommended for most CI/CD workloads.

Example configuration for networking:

toml [[runners]] (...) executor = "docker" [runners.docker] network_mode = "bridge"

It is important to note that the GitLab Runner emulated link behavior differs from legacy container links. For instance, if inter-container communication (icc) is disabled, containers cannot communicate with one another. Furthermore, environment variables for linked containers, such as <name>_PORT_<port>_<protocol>, are no longer provided. While Docker updates the /etc/hosts file in the container with the service container hostname and alias for name resolution, the service container itself cannot resolve the name of the main build container.

Performance Optimization via Tmpfs

To optimize I/O performance, particularly for database-heavy workloads, GitLab Runner supports the use of tmpfs. This allows the mounting of specific directories in RAM, significantly reducing the time required for tests that perform heavy disk operations. This is configured within the config.toml file under the [runners.docker] and [runners.docker.services_tmpfs] sections.

For example, to mount the MySQL data directory in RAM for both the main container and the services, the following configuration is used:

```toml
[runners.docker]

For the main container

[runners.docker.tmpfs]
"/var/lib/mysql" = "rw,noexec"

For services

[runners.docker.services_tmpfs]
"/var/lib/mysql" = "rw,noexec"
```

Additionally, GitLab Runner ensures that a /builds directory is mounted to all shared services, facilitating the exchange of data between the primary job and the service containers.

Private Registry Authentication and Credential Helpers

When using images from private registries, such as Amazon ECR, the runner requires specific authentication mechanisms. If a user attempts to use both a private registry and public images from Docker Hub without a credential helper, the process may fail because the Docker daemon tries to apply the same credentials to all registries.

To correctly configure access for a private registry, such as <aws_account_id>.dkr.ecr.<region>.amazonaws.com/private/image:latest, the following steps must be taken:

  1. Ensure the docker-credential-ecr-login helper is present in the GitLab Runner $PATH.
  2. Setup the necessary AWS credentials so the GitLab Runner Manager can acquire and pass them to the runners.
  3. Configure the runner to use the helper.

This can be achieved via two methods:

  • Creating a CI/CD variable named DOCKER_AUTH_CONFIG with the value { "credsStore": "osxkeychain" }.
  • For self-managed runners, adding the JSON configuration directly to ${GITLAB_RUNNER_HOME}/.docker/config.json.

The credsStore value allows the runner to access all registries using the designated helper.

Windows-Based Docker Execution

Running the Docker executor on Windows Server requires a recent version of Docker. A critical compatibility note is that Docker 17.06 is known to be incompatible with GitLab Runner because it fails to identify the Windows Server version, resulting in the error: unsupported Windows Version: Windows Server Datacenter.

A typical configuration for a Windows Docker executor looks as follows:

toml [[runners]] name = "windows-docker-2019" url = "https://gitlab.com/" token = "xxxxxxx" executor = "docker-windows" [runners.docker] image = "mcr.microsoft.com/windows/servercore:1809_amd64" volumes = ["c:\\cache"]

There is a known issue when a runner is registered with c:\\cache as a source directory via --docker-volumes or the DOCKER_VOLUMES environment variable.

To support different Windows versions and PowerShell requirements, GitLab provides specific helper images:

  • gitlab/gitlab-runner-helper:x86_64-vXYZ-nanoserver21H2
  • gitlab/gitlab-runner-helper:x86_64-vXYZ-servercore21H2
  • gitlab/gitlab-runner-helper:x86_64-vXYZ-nanoserver1809
  • gitlab/gitlab-runner-helper:x86_64-vXYZ-servercore1809

Notably, due to Windows container backward compatibility, Windows Server 2025 (24H2) is capable of using the 21H2 (Windows Server 2022) helper images.

Deployment, Logging, and Security

The method of starting the GitLab Runner influences how logs are captured. When run as a foreground task (either as a binary or in a container), logs are printed to stdout. When run as a system service via systemd, logs are handled by the system logging mechanism like Syslog. For Docker-based services, the docker logs command is used.

For example, if a container is started with:

docker run -d --name gitlab-runner --restart always -v /var/run/docker.sock:/var/run/docker.sock -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:latest

The logs can be viewed by executing:

docker logs gitlab-runner

Regarding security, if the GitLab CI/CD server uses self-signed SSL certificates, the runner container must trust the server certificate to avoid communication failures. The root certificates should be contained in a ca.crt file. By default, the gitlab/gitlab-runner image searches for this file in /etc/gitlab-runner/certs/ca.crt. This path can be modified using the environment variable:

-e "CA_CERTIFICATES_PATH=/DIR/CERT"

Conclusion

The integration of GitLab Runner with Docker transforms the CI/CD pipeline into a highly scalable and reproducible engine. By utilizing the Docker executor, developers can define precise environments via images, optimize performance using tmpfs for database operations, and maintain security through strict SSL certificate management and credential helpers for private registries. The flexibility extends from Linux-based Alpine and Ubuntu images to specialized Windows Server Core and NanoServer helper images, ensuring that the runner can adapt to virtually any software stack. The critical nature of the $PATH for credential helpers and the necessity of base utilities like sh and grep highlight the dependency of the runner on the underlying container's configuration. Ultimately, the shift toward a containerized runner architecture mitigates the "works on my machine" problem by ensuring that every job is executed in a clean, ephemeral, and precisely defined environment.

Sources

  1. Using Docker images
  2. Install GitLab Runner on Docker
  3. GitLab Runner Docker executor

Related Posts