Orchestrating Containerized Deployments via GitLab CI and Docker

The integration of GitLab CI/CD with Docker represents a pivotal shift in modern software engineering, transitioning from manual server configuration to a fully automated pipeline where code transforms into a running service without human intervention. GitLab, as an open-source Git code management system, functions similarly to platforms like GitHub or Bitbucket but distinguishes itself through a deeply integrated Continuous Integration and Continuous Deployment (CI/CD) engine. This native integration allows developers to define the entire lifecycle of an application—from building the image and running tests to deploying onto production clusters—within a single ecosystem.

The core of this automation is the GitLab Runner, an agent that executes the jobs defined in the .gitlab-ci.yml configuration file. When deploying Dockerized applications, the Runner can be configured in various modes depending on the target environment, whether it be a standalone Docker host, a Docker Swarm mode cluster, or an isolated build machine. The architectural flexibility of GitLab CI allows for a hybrid approach: building images on isolated, high-performance runners to ensure security and stability, and then deploying those images to a production Docker Swarm cluster using a specialized runner configured within that same cluster. This separation of concerns prevents production environments from being bogged down by resource-intensive build processes while ensuring that the deployment mechanism has the necessary privileges to orchestrate containers.

GitLab Runner Configuration and Executor Strategies

To execute Docker commands within a CI/CD pipeline, the GitLab Runner must be specifically configured to interact with the Docker Engine. The choice of "executor" determines how the job is isolated and how it accesses the underlying host.

The Shell Executor is a primary method for enabling Docker commands. In this configuration, the gitlab-runner user executes commands directly on the host machine's shell. For this to function, the Docker Engine must be installed on the server where the runner resides, and the gitlab-runner user must be granted the necessary permissions to execute Docker commands, typically by being added to the docker group.

The registration process for a shell executor typically follows this command structure:

sudo gitlab-runner register -n --url "https://gitlab.com/" --registration-token REGISTRATION_TOKEN --executor shell --description "My Runner"

Alternatively, the Docker Executor allows CI/CD jobs to run inside Docker containers. This provides a clean, ephemeral environment for every job. To use this, the runner is registered with the Docker executor, and the .gitlab-ci.yml file specifies the image required for the job. This approach can include optional services, such as MySQL, running in sidecar containers to facilitate integration testing.

For those utilizing Docker Swarm mode, running the GitLab Runner in Docker standalone mode is often preferred, even when deploying to a Swarm Manager Node. This is because runner configurations persist in the created container after the registration process, ensuring stability across restarts and updates.

Implementing Docker-in-Docker (DinD) for Image Construction

A common challenge in CI/CD is the need to build a Docker image inside a container. This is achieved through Docker-in-Docker (DinD). This method requires the runner to operate in privileged mode, which grants the container access to the host's kernel features necessary to run the Docker daemon.

A typical pipeline utilizing DinD consists of a build stage where the image is created and pushed to a registry. The following configuration illustrates the implementation:

```yaml
stages:
- build
- deploy

build:
stage: build
tags:
- dind
image: docker:20.10.16
services:
- docker:20.10.16-dind
variables:
DOCKERTLSCERTDIR: "/certs"
beforescript:
- docker login -u $CIREGISTRYUSER -p $CIREGISTRYPASSWORD $CIREGISTRY
script:
- docker pull $CIREGISTRYIMAGE:latest || true
- docker build --cache-from $CIREGISTRYIMAGE:latest --tag $CIREGISTRYIMAGE:$CICOMMITSHA --tag $CIREGISTRYIMAGE:latest .
- docker push $CIREGISTRYIMAGE:$CICOMMITSHA
- docker push $CIREGISTRYIMAGE:latest
```

In this workflow, the DOCKER_TLS_CERTDIR variable is critical for securing the communication between the Docker client and the DinD service. The before_script ensures authentication with the GitLab Container Registry using predefined environment variables. The docker build command uses --cache-from to optimize build times by leveraging previously pushed images.

Continuous Deployment with Docker Compose and SSH

For environments that do not use Swarm but rely on Docker Compose for orchestration, a different deployment strategy is required. This involves using the GitLab pipeline to update a docker-compose.yml file on a remote server and triggering a restart via SSH.

To establish this secure connection, a dedicated deployment user should be created on the target server:

sudo adduser <project>
sudo adduser <project> docker

Following the user creation, an SSH key pair must be generated to allow the GitLab Runner to authenticate without a password:

sudo su <project>
cd ~
ssh-keygen -t rsa -b 4096
cat .ssh/id_rsa.pub > .ssh/authorized_keys

The private key is then stored as a protected variable in the GitLab web interface under Settings > CI/CD > Variables with the name SSH_PRIVATE_KEY. To prevent "man-in-the-middle" attacks, the server's SSH fingerprint must be captured using ssh-keyscan:

ssh-keyscan -t rsa -H <deploy.server>

This fingerprint is stored in another GitLab variable named SSH_HOST_KEY. This setup allows the pipeline to securely transfer the docker-compose.yml file and execute restart commands on the remote host.

Docker Swarm Orchestration and Deployment

Deploying to a Docker Swarm cluster requires the Runner to have access to the Docker socket of a Swarm Manager node. This is typically achieved by mounting the socket during runner registration:

--docker-volumes /var/run/docker.sock:/var/run/docker.sock --url $GITLAB_URL --registration-token $GITLAB_TOKEN --tag-list dog-cat-cluster,stag,prod

By mounting /var/run/docker.sock, the Runner can issue commands directly to the Swarm manager. A deployment pipeline for Swarm typically uses the docker stack deploy command.

Example Swarm deployment configuration:

```yaml
image: tiangolo/docker-with-compose
beforescript:
- docker login -u gitlab-ci-token -p $CIJOBTOKEN $CIREGISTRY
stages:
- build
- deploy

build-prod:
stage: build
script:
- docker-compose build
only:
- master

deploy-prod:
stage: deploy
script:
- docker stack deploy -c docker-compose.yml --with-registry-auth my-stack
only:
- master
```

The --with-registry-auth flag is essential in Swarm deployments as it ensures that the worker nodes in the cluster can pull the image from the private GitLab registry using the credentials provided during the deployment phase.

Advanced Registry Authentication and Credential Helpers

Managing access to private registries, especially those outside of GitLab such as Amazon ECR, requires specific configuration of the GitLab Runner's environment.

For standard Docker configuration, the DOCKER_AUTH_CONFIG variable can be used to store the JSON content of the Docker config file:

{ "credsStore": "osxkeychain" }

In self-managed runner environments, this JSON can be placed directly into ${GITLAB_RUNNER_HOME}/.docker/config.json.

When using external registries like AWS ECR, the docker-credential-ecr-login helper must be available in the Runner's $PATH. The process involves:

1. Ensuring the credential helper binary is installed.
2. Configuring AWS credentials that the GitLab Runner Manager can access and pass to the runners.
3. Configuring the Runner to utilize the helper for images following the pattern <aws_account_id>.dkr.ecr.<region>.amazonaws.com.

This is critical because if a pipeline uses both public images from Docker Hub and private images from a registry, the Docker daemon may fail to pull from Docker Hub if it attempts to use the private registry's credentials for all requests.

Comparison of Runner Execution Methods

The following table provides a technical comparison of the various methods used to execute Docker commands within GitLab CI.

Comprehensive Workflow Analysis

The transition from a code commit to a production deployment involves several interlocking stages, each with its own failure points and requirements.

The Build Stage is the most resource-intensive. Utilizing the docker build command with --cache-from is not merely an optimization but a necessity for maintaining fast pipeline turnaround times. By pulling the latest image and using it as a cache source, the runner avoids rebuilding layers that have not changed.

The Deployment Stage varies by target. In the Docker Compose scenario, the pipeline acts as a remote controller, using SSH to push a manifest and restart services. In the Docker Swarm scenario, the pipeline acts as an orchestrator, telling the Swarm Manager to update the desired state of the stack.

The security layer is maintained through the use of GitLab CI/CD variables. Sensitive data such as SSH_PRIVATE_KEY and CI_REGISTRY_PASSWORD are never hardcoded in the .gitlab-ci.yml file. This prevents credential leakage in the event that the source code is compromised.

Conclusion

The synergy between GitLab CI and Docker creates a robust framework for continuous delivery, allowing for a high degree of automation and reliability. By utilizing a combination of the Docker executor for isolation and the Shell executor for host-level management, organizations can build secure, scalable pipelines. The use of Docker-in-Docker provides the necessary environment for image creation, while the integration with Docker Swarm enables sophisticated orchestration across multiple nodes. The critical success factor in these deployments lies in the correct configuration of the GitLab Runner, the strategic use of the Docker socket for orchestration, and the rigorous management of registry credentials through specialized helpers and protected variables. This architecture ensures that the path from a developer's commit to a production container is seamless, repeatable, and secure.

Sources

1. DockerSwarm.rocks
2. CyLab Blog
3. GitLab Documentation - Build Docker Images
4. GitLab Documentation - Using Docker Images