The implementation of a continuous integration and continuous deployment (CI/CD) pipeline requires a robust execution environment capable of processing build tasks, running automated tests, and deploying software packages. In the GitLab ecosystem, this functionality is handled by the GitLab Runner, an application specifically designed to process builds as specified in a YAML configuration file. When deploying these runners on DigitalOcean, organizations can transition from static, single-server environments to highly elastic, cloud-scalable architectures. By utilizing the Docker executor and the docker+machine configuration, developers can create a system where the infrastructure expands and contracts based on the actual demand of the build queue, ensuring that resources are only consumed when active jobs are being processed.
This architectural approach separates the control plane from the execution plane. The control instance, often referred to as the bastion server, acts as the orchestrator. It interacts with the DigitalOcean API to manage the lifecycle of transient droplets. These transient droplets, or GitLab Runners, are disposable servers created on the fly to execute a specific job. Once the job is completed and the specified idle time has elapsed, these servers are destroyed, preventing the accumulation of unused resources and optimizing cost efficiency.
DigitalOcean Droplet Provisioning for GitLab Runners
The initial setup of a GitLab Runner on DigitalOcean can be achieved through several methods, ranging from manual installation on a clean image to utilizing pre-configured one-click applications.
For those seeking a rapid deployment, DigitalOcean provides a one-click image with Docker pre-installed on Ubuntu 14.04. This streamlines the process by eliminating the manual installation of the container engine. To initiate this, a user must log into the DigitalOcean Control Panel and select the Create Droplet button. Within the Choose an image section, the One-click Apps tab allows the selection of the Docker image.
Resource allocation is a critical decision during this phase. While various sizes are available, a droplet with 1 GB of RAM and 1 CPU is generally advised for quicker builds, providing a balanced cost-to-performance ratio for small to medium CI/CD tasks. After selecting the region and adding the necessary SSH keys, the droplet is provisioned.
Upon gaining SSH access to the new droplet, the installation of Docker must be verified. This is performed using the following command:
docker info
This command outputs detailed information regarding the Docker version and the current count of images and containers, confirming that the environment is ready for the GitLab Runner installation. While the one-click image is a convenient path, GitLab Runner is supported across all operating systems and various Linux distributions, including Debian, Ubuntu, and CentOS. Users are also free to utilize FreeBSD if supported by the DigitalOcean environment.
Manual Runner Deployment via Docker-in-Docker
For advanced configurations, such as those requiring a custom Docker-in-Docker (DinD) approach, a more manual setup is utilized. This method involves the use of Docker Machine to provision specific nodes.
The process begins by configuring the environment with the DigitalOcean access token:
export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]
With Docker Machine installed on the local machine, a dedicated runner node is created. The following command exemplifies the creation of a droplet named runner-node in the nyc1 region using a Debian 10 image and a larger resource profile (4 vCPUs and 8 GB of RAM) to handle more intensive workloads:
bash
docker-machine create \
--driver digitalocean \
--digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
--digitalocean-region "nyc1" \
--digitalocean-image "debian-10-x64" \
--digitalocean-size "s-4vcpu-8gb" \
--engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
runner-node
Once the node is active, the user must SSH into the droplet using:
docker-machine ssh runner-node
The deployment then requires the creation of a specific directory structure and configuration files:
- config/config.toml
- docker-compose.yml
The docker-compose.yml file is used to orchestrate the runner container. A standard configuration uses version 3 of the Compose specification and employs the official GitLab Runner Docker image (e.g., v14.3.2). The configuration is as follows:
yaml
version: '3'
services:
gitlab-runner-container:
image: gitlab/gitlab-runner:v14.3.2
container_name: gitlab-runner-container
restart: always
volumes:
- ./config/:/etc/gitlab-runner/
- /var/run/docker.sock:/var/run/docker.sock
In this setup, the Docker socket is mounted to allow the runner to manage other containers, and the config folder is persisted. Port 9252 is exposed to the Docker host to facilitate communication.
Autoscaling Configuration and the Bastion Server
The transition from a static runner to an autoscaling environment requires the configuration of a bastion server. The bastion server does not execute any build jobs itself; instead, it serves as the control instance that communicates with the DigitalOcean API to spawn and destroy worker droplets.
The primary configuration for this behavior is located in the config.toml file, which can be edited using a text editor such as nano:
nano /etc/gitlab-runner/config.toml
The config.toml file defines the rules for scaling. To enable autoscaling, the docker+machine executor must be specified. The following configuration parameters are essential for a functioning autoscaling setup:
| Parameter | Value/Description | Impact |
|---|---|---|
| concurrent | 50 | Defines the maximum number of concurrent builds across all registered runners. |
| executor | docker+machine | Tells the runner to use Docker Machine to create new instances. |
| limit | 10 | The maximum number of created machines this specific runner can manage. |
| IdleCount | 1 | Maintains one machine in a ready state even if the queue is empty. |
| IdleTime | 600 | Machines are destroyed after 600 seconds of inactivity. |
| MachineDriver | digitalocean | Specifies the use of the DigitalOcean cloud driver. |
The MachineOptions block is where the specific DigitalOcean environment is defined. This ensures that every transient droplet created follows the same specifications:
- digitalocean-image=coreos-stable
- digitalocean-ssh-user=core
- digitalocean-access-token=DOACCESSTOKEN
- digitalocean-region=nyc3
- digitalocean-size=1gb
- digitalocean-private-networking
The use of a unique MachineName such as gitlab-runner-autoscale-%s is mandatory, as the %s placeholder allows the system to generate random numbers to distinguish between multiple active droplets. Additionally, the setup can utilize a distributed cache, such as an S3-compatible cache, to maintain build speeds across transient instances.
Operational Maintenance and Troubleshooting
Managing a cloud-scalable CI/CD pipeline involves monitoring the lifecycle of the droplets to prevent unexpected billing and ensuring the runner remains reachable.
Because the runners are transient, they should be automatically destroyed after the IdleTime expires. However, it is recommended that administrators manually verify that droplets are being deleted in the DigitalOcean dashboard to avoid cost overruns. In scenarios where the configuration causes an excessive number of machines to be created, they can be purged simultaneously using the following command:
docker-machine rm $(docker-machine ls -q)
If the GitLab instance reports that a runner is unreachable, the system may fail to deploy new runners. This often indicates a configuration error or a network partition. Troubleshooting can be performed by stopping the runner and restarting it in debug mode to capture detailed error logs:
gitlab-runner stop
gitlab-runner --debug start
The debug output provides specific error messages that help identify whether the issue lies in the API token, the region settings, or the Docker Machine driver configuration.
Advanced Optimization Strategies
Once the basic autoscaling pipeline is established, further performance optimizations can be implemented to reduce build times and resource waste.
One primary method is the integration of a Docker Registry for higher levels of caching. By caching intermediate layers of Docker images, the time required to pull images onto new transient droplets is significantly reduced. Furthermore, the use of tagging allows developers to route specific code builds to specialized runners. For example, a build requiring a GPU for machine learning tasks can be tagged to only run on a runner node with the appropriate hardware specifications, while standard web tasks remain on the 1 GB droplets.
The use of the docker+machine executor ensures that the environment is always clean, as each job starts with a fresh image, eliminating the "snowflake server" problem where build artifacts from previous jobs interfere with current ones.
Conclusion
The deployment of GitLab Runners on DigitalOcean transforms the CI/CD process from a static bottleneck into a dynamic, responsive resource. By leveraging a bastion server and the docker+machine executor, organizations can achieve a state of "cloud scalability" where the infrastructure precisely mirrors the demand of the development cycle. The transition from a single Docker-enabled droplet to a fleet of transient, disposable servers allows for massive concurrency—up to 50 builds in the provided configuration—without sacrificing the stability of the primary control plane.
The critical nature of the config.toml file cannot be overstated, as it governs not only the performance (via concurrent and limit settings) but also the financial expenditure (via IdleTime and digitalocean-size). The integration of Docker-in-Docker and the use of specific DigitalOcean images like CoreOS ensure that the execution environment is optimized for containerization. Ultimately, the ability to programmatically create and destroy infrastructure via the DigitalOcean API removes the manual overhead of server maintenance, allowing engineering teams to focus on code delivery rather than infrastructure management.