Orchestrating GitLab Runner Infrastructure via Azure Virtual Machine Scale Sets

The integration of GitLab and Microsoft Azure represents a strategic convergence of source code management and cloud computing, designed to eliminate the bottlenecks associated with traditional software delivery. GitLab, as a comprehensive web-based Git repository manager, provides the essential framework for version control, issue tracking, and CI/CD pipeline definition. When paired with Azure—a cloud platform offering expansive virtual machine (VM) capabilities, scalable storage, and sophisticated networking—organizations can transition from static build environments to a dynamic, elastic infrastructure. This synergy allows development teams to leverage the scalability and reliability of the cloud to accelerate the software delivery lifecycle, ensuring that compute resources are available exactly when a pipeline triggers and are released immediately upon completion to optimize cost and performance.

At the heart of this automation is the GitLab Runner. A runner is a specialized piece of software designed to execute jobs defined in a project's CI/CD configuration. When a pipeline is triggered—whether by a manual action or an automated commit—the runner polls the GitLab instance, receives the job instructions, and executes the specified scripts. The architectural flexibility of runners is a primary driver for adopting Azure, as it allows for the deployment of different runner types based on the specific needs of the project.

Shared runners are those provided by GitLab itself and are accessible to all projects within an instance, offering a convenient but often resource-constrained environment. In contrast, specific runners are dedicated to a single project and are defined within the project’s .gitlab-ci.yml file, providing a controlled environment where the user has full authority over the hardware and software configuration. Furthermore, group runners serve as a middle ground, providing a set of runners shared across multiple projects within a specific group. For high-resource jobs or large-scale deployments, the Kubernetes executor is often utilized, as it is specifically engineered for horizontal scaling and the orchestration of containerized workloads.

Architectural Framework of Azure Virtual Machine Scale Sets (VMSS)

The deployment of a GitLab Runner on Azure via Virtual Machine Scale Sets (VMSS) introduces a sophisticated auto-scaling mechanism that moves beyond the limitations of a single virtual machine. This architecture utilizes Infrastructure as Code (IaC) templates and scripts to ensure that the environment is reproducible and scalable.

The operational logic of the VMSS-based runner is governed by a specific sequence of events:

• Manager VM Operation: A central Manager VM is deployed to run the GitLab Runner software using the "instance" executor and the Azure autoscaler plugin. This VM acts as the brain of the operation.
• Registration Process: The manager VM uses a secure registration token to authenticate and register itself with the specific GitLab instance.
• Continuous Job Polling: The manager VM does not execute the jobs itself but continuously polls the GitLab instance to check for pending jobs in the queue.
• Dynamic Auto-scaling: Once the manager detects that jobs are queued, the Azure autoscaler plugin triggers the creation of new instances within the Virtual Machine Scale Set.
• Job Distribution: The actual workload is distributed across the available VMSS instances, allowing multiple jobs to run in parallel.
• Resource Reclamation: To maintain cost efficiency, the system implements a scale-down policy. After the jobs are completed and a predefined idle time has expired, the VMSS instances are automatically terminated.

Prerequisites for Deployment

Before initiating the deployment of the GitLab Runner infrastructure on Azure, several technical requirements must be met to ensure a seamless installation and connectivity.

The following table outlines the mandatory requirements for a successful setup:

Securing the Runner Registration Token

A critical security step in this process is the acquisition and protection of the runner registration token. This token acts as the secret handshake between the Azure VM and the GitLab instance; if compromised, unauthorized runners could be attached to the project.

To obtain the token, the following sequence must be followed:

• Navigate to the specific GitLab project URL, for example, https://gitlab.com/your-username/your-project.
• Access the project settings by going to Settings and then selecting CI/CD.
• Expand the "Runners" section to view the available configuration options.
• Click the "New project runner" button to initiate the creation of a new runner entity.
• Configure the runner settings by selecting the operating system, such as Linux or Windows.
• Assign optional tags, such as azure or vmss, which allow the .gitlab-ci.yml file to target this specific runner for specialized jobs.
• Determine whether the runner should pick up all jobs by checking the "Run untagged jobs" option.
• Finalize the process by clicking "Create runner" and copying the displayed registration token, which begins with the prefix glrt-.

Deployment Procedures and Technical Execution

The deployment can be achieved through two primary methods: a manual installation on a standalone Azure VM or an automated deployment using the VMSS sample repository.

Automated VMSS Deployment

For those seeking a scalable, production-ready environment, the Azure-Samples repository provides the necessary IaC templates.

The execution begins with cloning the official sample repository and navigating into the directory:

bash git clone https://github.com/Azure-Samples/Gitlab-Runner-VMSS.git cd Gitlab-Runner-VMSS

Following the cloning process, the user applies the provided templates, integrating the registration token obtained from GitLab. This process automates the creation of the Manager VM and the associated Scale Set.

Manual Installation on Azure VM

For simpler requirements or testing, a runner can be installed manually on a Debian-based Azure virtual machine.

The installation of the GitLab Runner binary is performed using the official repository script:

bash curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bas sudo apt-get install gitlab-runnerh

Once the software is installed, the registration process is initiated to link the VM to the GitLab instance:

bash gitlab-runner register

During the registration prompt, the user must provide the GitLab instance URL and the registration token. These values are located in the GitLab instance under Admin Area > Overview > Runners. After providing the description and tags, the runner service is started:

bash gitlab-runner start

Validation and Verification of Connectivity

After the deployment is complete, it is imperative to verify that the runner is operational and communicating with the GitLab instance.

The first step is to check the GitLab UI. Navigate to the Admin Area > Overview > Runners. A successful deployment will show the runner listed with a green "online" indicator. In the case of the VMSS deployment, the runner description will typically follow the naming convention azure-vmss-runner-{environment-name}.

To practically test the functionality, a test pipeline should be created. This involves creating a .gitlab-ci.yml file in the project root with the following content:

yaml test-runner: script: - echo "Hello from Azure VMSS runner!" - uname -a

Upon committing and pushing this file, the user should navigate to CI/CD > Pipelines to confirm that the job is being picked up and executed by the Azure-based runner.

Monitoring and Troubleshooting the Azure Infrastructure

Effective management of the GitLab Runner requires consistent monitoring of the Azure resources to ensure there are no deployment failures or network bottlenecks.

Users can access the Azure Portal to monitor the deployment by using the Azure CLI to open the specific resource group:

bash az group show --name rg-{your-environment-name} --query id -o tsv | xargs -I {} open "https://portal.azure.com/#@/resource{}"

Within the Azure Portal, the following components must be verified:

• The Manager VM status must be "Running".
• The VMSS must be present, though it may show 0 instances if no jobs are currently queued.
• Network Security Groups (NSGs) must be correctly configured to allow necessary traffic.
• The Activity Log should be reviewed for any deployment errors.

If the runner does not appear online in GitLab, a deeper technical investigation is required. This involves SSHing into the manager VM using the IP address found in the Azure Portal:

bash ssh azureuser@{manager-vm-ip}

Once inside the VM, the following diagnostic commands should be executed to verify the status of the runner and the underlying system:

bash sudo gitlab-runner verify sudo systemctl status gitlab-runner

To examine the real-time logs of the runner for any errors during job polling or instance creation, use the journalctl command:

bash sudo journalctl -u gitlab-runner -f

Furthermore, network connectivity to the GitLab instance must be validated from within the VM to ensure that firewalls are not blocking the connection:

bash curl -I https://gitlab.com

Finally, the Azure deployment logs should be reviewed by navigating to the resource group in the Azure Portal and selecting "Deployments" in the left-hand menu to identify any failed infrastructure updates.

Conclusion

The implementation of a GitLab Runner on Azure, particularly when leveraging Virtual Machine Scale Sets, transforms the CI/CD pipeline from a static cost center into a dynamic asset. By utilizing the "instance" executor and the Azure autoscaler plugin, organizations can achieve a state of operational efficiency where compute resources are scaled horizontally in direct response to demand. This eliminates the "noisy neighbor" effect and resource contention common in shared runners, while avoiding the wasted expenditure of permanently running large VMs.

The transition from manual installation—using apt-get and gitlab-runner register—to a fully automated IaC approach via the Gitlab-Runner-VMSS repository signifies a shift toward modern DevOps practices. The ability to programmatically deploy, scale, and terminate runners based on pipeline requirements ensures that software delivery is not only accelerated but also optimized for cost and reliability. Whether employing a single Azure VM for small projects or a complex VMSS architecture for enterprise-grade workloads, the integration provides a robust foundation for continuous integration and continuous deployment in the cloud.

Sources

1. Understanding GitLab Runner - Nikila Fernando
2. Gitlab-Runner-VMSS - Azure Samples GitHub