Architectural Paradigms for Containerized Image Construction within GitLab Runner on Kubernetes

The orchestration of CI/CD pipelines within a Kubernetes ecosystem introduces a fundamental paradox in containerized workflows: the necessity of building container images requires a container engine, yet running a container engine inside a containerized environment introduces significant security, stability, and architectural complexities. When utilizing GitLab Runner on a Kubernetes cluster, engineers face a critical decision regarding the methodology used to execute docker build commands. The traditional approach of nesting Docker engines—commonly referred to as Docker-in-Docker (DinD)—requires elevated privileges and complex configuration of TLS certificates and network sockets. Conversely, modern alternatives like Kaniko provide a daemonless path to image construction, eliminating the need for privileged containers entirely. This technical analysis examines the mechanisms, configurations, and security implications of implementing Docker-in-Docker, Docker socket binding, and Kaniko within a Kubernetes-based GitLab Runner environment.

The Docker-in-Docker (DinD) Methodology

Docker-in-Docker (DinD) refers to a configuration where a GitLab Runner, utilizing either the Docker or Kubernetes executor, spins up a container that contains its own fully functional Docker daemon. In a Kubernetes context, this means the job pod created by the GitLab Runner includes a sidecar service container running the Docker daemon, allowing the primary job container to issue commands to it.

Implementing DinD requires a specific orchestration of images and services. The job container typically utilizes a Docker CLI image, while a secondary service container provides the daemon. For instance, when using version-pinned images, the configuration might involve docker:24.0.5-cli for the job and docker:24.0.5-dind for the service.

Configuration Requirements and TLS Implementation

Modern Docker versions, specifically from 19.03.12 onwards, support connections over Transport Layer Security (TLS) by default. In a GitLab CI/CD pipeline, this is managed through the DOCKER_TLS_CERTDIR environment variable. When this variable is set, Docker facilitates secure communication between the client and the daemon using shared certificates.

The standard configuration for a TLS-enabled DinD job in a .gitlab-ci.yml file is structured as follows:

yaml build_image: image: docker:24.0.5-cli services: - name: docker:24.0.5-dind variables: DOCKER_TLS_CERTDIR: "/certs"

The impact of using TLS is a significant increase in the security posture of the inter-container communication, preventing unauthorized interception of commands. However, it adds complexity to the environment setup, as both the client and the service must recognize the same certificate directory.

Disabling TLS for Simplified Environments

In certain testing or isolated scenarios, users may choose to disable TLS to reduce configuration overhead. This is achieved by setting DOCKER_TLS_CERTDIR to an empty string. When TLS is disabled, the client must be instructed to communicate with the daemon via a standard TCP connection rather than a secure socket.

To execute a non-TLS build, the following configuration is required:

```yaml
default:
image: docker:24.0.5-cli
services:
- name: docker:24.0.5-dind
variables:
DOCKERTLSCERTDIR: ""
DOCKERHOST: tcp://docker:2375
HEALTHCHECKTCP_PORT: "2375"

build:
stage: build
tags:
- no-tls-dind-kubernetes-runner
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests
```

The use of DOCKER_HOST: tcp://docker:2375 is critical here; it directs the Docker CLI to look for the daemon at the service's network alias rather than the default /var/run/docker.sock. While this simplifies the certificate management, it removes the encryption layer for the communication between the job and the daemon.

Kubernetes Executor Configuration for DinD

When the GitLab Runner is configured with the Kubernetes executor, the administrator must modify the config.toml of the runner to allow for privileged execution. Without the privileged = true flag, the DinD service container will fail to start the Docker daemon, as the daemon requires access to specific kernel capabilities.

The runner configuration for a Kubernetes executor using DinD follows this pattern:

toml [[runners]] name = "kubernetes-dind-runner" executor = "kubernetes" [runners.kubernetes] image = "ubuntu:20.04" privileged = true [[runners.kubernetes.services]] name = "docker:24.0.5-dind" command = ["--registry-mirror", "https://registry-mirror.example.com"]

The inclusion of the --registry-mirror command allows the daemon to optimize image pulling by using a local or preferred mirror, which is essential for avoiding Docker Hub rate limits and improving build performance.

Docker Socket Binding on Kubernetes

Docker socket binding is an alternative to DinD that avoids running a separate daemon inside the pod. Instead, it mounts the host machine's Docker socket (/var/run/docker.sock) directly into the GitLab Runner's job container. This allows the container to issue commands that are executed directly by the host's Docker engine.

Implementation via Host Path Volumes

To implement socket binding on Kubernetes, the GitLab Runner must be configured to use host_path volumes. This provides the container with a direct link to the underlying node's Docker daemon.

The following configuration fragment demonstrates how to map the socket in the config.toml:

toml [[runners]] name = "socket-binding-kubernetes-runner" executor = "kubernetes" [runners.kubernetes] image = "ubuntu:20.04" privileged = false [[runners.kubernetes.volumes.host_path]] host_path = '/var/run/docker.sock' mount_path = '/var/run/docker.sock' name = 'docker-sock' read_only = true

In the corresponding .gitlab-ci.yml file, the job can then proceed as if a local Docker daemon were present:

```yaml
default:
image: docker:24.0.5-cli
before_script:
- docker info

build:
stage: build
tags:
- socket-binding-kubernetes-runner
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests
```

Security Implications and Risk Assessment

While socket binding allows for the avoidance of privileged = true in many contexts, it introduces severe security vulnerabilities. By mounting the Docker socket, the container effectively gains root-level control over the host machine.

The following risks are inherent to this method:

• Privilege Escalation: A malicious or compromised job script can execute commands like docker rm -f $(docker ps -a -q), which would terminate all running containers on the host, including the GitLab Runner itself.
• Container Breakout: The ability to interact with the host daemon allows for the creation of containers that can escape their isolation and access the host's filesystem or network.
• Resource Conflict: Since all jobs share the same host daemon, concurrent jobs may experience conflicts if they attempt to create containers with identical names.
• Sibling Relationship: Containers created via socket binding are siblings of the runner on the host, rather than children of the runner pod, which breaks the expected lifecycle isolation of Kubernetes.

Kaniko: The Daemonless Alternative

Kaniko offers a paradigm shift in how container images are built within Kubernetes. Unlike DinD or socket binding, Kaniko does not require a Docker daemon or privileged access. It executes every build step entirely in userspace, making it an ideal candidate for highly secure, multi-tenant Kubernetes clusters.

Operational Advantages of Kaniko

The primary advantage of Kaniko is its ability to operate within a standard, unprivileged container. This eliminates the "security vs. functionality" trade-off inherent in DinD. Because Kaniko does not rely on a daemon, it is not susceptible to the host-level risks associated with socket binding or the kernel requirement risks of DinD.

Kaniko's workflow involves:
- Extracting the filesystem of the base image.
- Executing each command in the Dockerfile within a specialized userspace environment.
- Capturing the changes to the filesystem.
- Pushing the resulting image directly to a container registry.

Registry Authentication and Configuration

For Kaniko to push images to a private registry, it must have access to the necessary credentials. Kaniko automatically looks for Docker configuration files in a specific directory: /kaniko/.docker/config.json.

To facilitate this, users can provide a configuration file containing the registry credentials. In a GitLab CI/CD pipeline, this is often handled by creating the configuration file dynamically using variables provided by the GitLab environment.

An example of a Kaniko execution command in a .gitlab-ci.yml file would look like this:

yaml build: stage: build image: name: gcr.io/kaniko-project/executor:debug entrypoint: [""] script: - mkdir -p /kaniko/.docker - echo "{\"auths\":{\"my-registry.example.com\":{\"auth\":\"$(echo -n $username:$password | base64)\"}}}" > /kaniko/.docker/config.json - /kaniko/executor --context "${CI_PROJECT_DIR}" --dockerfile "${CI_PROJECT_DIR}/Dockerfile" --destination "my-registry.example.com/my-image:${CI_COMMIT_TAG}"

The impact of this approach is a highly streamlined and secure pipeline. By utilizing the debug version of the Kaniko image, users gain access to a shell (like sh), which allows for the manual creation of the configuration directory and file before executing the /kaniko/executor binary.

Registry Mirroring and Daemon Optimization

Regardless of the chosen methodology, optimizing the interaction with container registries is crucial for CI/CD efficiency. Registry mirroring can significantly reduce build times by caching frequently used layers and images closer to the execution environment.

Configuring Mirrors via ConfigMaps

In a Kubernetes-based GitLab Runner, registry mirrors can be configured at the daemon level. For DinD, this involves mounting a daemon.json file into the runner's containers. This is best achieved using Kubernetes ConfigMaps.

1. Create a local file named daemon.json:
   json { "registry-mirrors": [ "https://registry-mirror.example.com" ] }

2. Create the ConfigMap in the appropriate namespace (e.g., gitlab-runner):
   bash kubectl create configmap docker-daemon --namespace gitlab-runner --from-file /tmp/daemon.json

3. Update the GitLab Runner config.toml to mount this ConfigMap:
   toml [[runners]] executor = "kubernetes" [runners.kubernetes] image = "alpine:3.12" privileged = true [[runners.kubernetes.volumes.config_map]] name = "docker-daemon" mount_path = "/etc/docker/daemon.json" sub_path = "daemon.json"

This configuration ensures that every job pod created by the runner automatically utilizes the specified registry mirror, providing a consistent performance boost across all pipelines.

Command-Line Mirroring for DinD Services

Alternatively, for pipelines using the DinD service, the mirror can be specified directly in the .gitlab-ci.yml file by passing flags to the service container's command.

yaml services: - name: docker:24.0.5-dind command: ["--registry-mirror", "https://registry-mirror.example.com"]

This method is useful for per-job customization, allowing specific pipelines to use different mirrors without requiring a global change to the GitLab Runner's configuration.

Comparative Analysis of Implementation Strategies

The selection of a build strategy depends on the intersection of security requirements, infrastructure control, and performance needs. The following table provides a technical comparison of the three primary methods discussed.

Technical Conclusion

The evolution of CI/CD orchestration within Kubernetes has moved away from the necessity of privileged, daemon-dependent workflows toward more isolated, secure models. While Docker-in-Docker (DinD) remains a robust solution for complex build scenarios requiring features like docker-compose, its reliance on privileged containers and TLS management introduces a significant attack surface and operational burden. Docker socket binding offers a high-performance alternative but at the cost of near-total host vulnerability, making it unsuitable for shared or production-grade Kubernetes environments.

Kaniko represents the modern standard for secure, cloud-native image construction. By operating in userspace and removing the requirement for a privileged daemon, it aligns perfectly with the principles of Kubernetes isolation and security. For organizations prioritizing a zero-trust architecture, the transition to Kaniko—despite the loss of certain Docker-specific CLI conveniences—is the most technically sound architectural decision. Ultimately, the choice must be dictated by the specific security constraints of the cluster and the complexity of the image build requirements.

Sources

1. Pachamamita: Building Docker Images with Kaniko on Kubernetes
2. Collabnix: GitLab Runner on Kubernetes
3. GitLab Documentation: Using Docker build in CI/CD