The orchestration of Continuous Integration and Continuous Deployment (CI/CD) workloads requires a robust, scalable, and reliable execution environment. In the modern DevOps paradigm, manual provisioning of runner instances is an anti-pattern that introduces configuration drift, human error, and significant operational overhead. By leveraging HashiCorp Terraform in conjunction with Amazon Web Services (AWS) and GitLab, engineering teams can implement Infrastructure as Code (IaC) to automate the lifecycle of GitLab Runners. This methodology ensures that the execution environment—specifically the EC2 instances tasked with running CI/CD jobs—is reproducible, version-controlled, and capable of being destroyed or scaled with minimal friction.

The integration of Terraform with GitLab provides a dual-layer automation benefit. First, Terraform manages the underlying cloud primitives, such as VPCs, Security Groups, and EC2 instances. Second, when integrated directly into GitLab's CI/CD pipelines, Terraform can manage its own state within the GitLab ecosystem using a remote HTTP backend. This creates a closed-loop system where the infrastructure used to build the software is managed by the same software development lifecycle (SDLC) that governs the application code itself.

Architectural Components of a Terraform-Driven GitLab Runner

To successfully deploy a GitLab Runner on AWS using Terraform, several distinct architectural layers must be orchestrated. Each component serves a specific role in ensuring the runner is reachable, secure, and correctly registered with the GitLab instance.

The infrastructure consists of the following primary entities:

Defining the Terraform Configuration Foundation

The deployment begins with the creation of a structured Terraform directory. This directory must contain the core configuration files that define the desired state of the AWS resources. Users can either clone an existing repository or initialize a new directory structure manually.

To prepare a new workspace, the following commands are utilized:

bash git clone [email protected]:TheDevOpsHub/TerraformHub.git cd TerraformHub/AWS/aws-ec2-gitlab-runner

Alternatively, a manual directory creation can be performed to ensure a clean slate:

bash mkdir -p TerraformHub/AWS/aws-ec2-gitlab-runner

Resource Definitions in main.tf

The main.tf file is the heart of the configuration. It defines the AWS Security Group and the EC2 instance itself. The Security Group must be configured with specific ingress and egress rules to allow for administrative access (SSH) while maintaining a restrictive posture.

The following configuration block defines the security group:

```hcl
resource "awssecuritygroup" "gitlab_runner" {
name = "gitlab-runner-sg"
description = "Security group for GitLab Runner"

ingress {
fromport = 22
toport = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}

egress {
fromport = 0
toport = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
```

The ingress rule for port 22 allows SSH access from any IP address (0.0.0.0/0), which is necessary for initial connection and troubleshooting. The egress rule is set to allow all outbound traffic (protocol = "-1"), enabling the runner to communicate with GitLab and download necessary packages via yum or curl.

The EC2 instance resource integrates the security group and utilizes a templatefile function to pass the GitLab registration token into the instance's initialization script.

hcl resource "aws_instance" "gitlab_runner" { ami = "ami-03972092c42e8c0ca" instance_type = var.instance_type key_name = var.key_name security_groups = [aws_security_group.gitlab_runner.name] user_data = templatefile("install_runner.tpl", { gitlab_runner_registration_token = var.gitlab_runner_registration_token }) tags = { Name = "AWS EC2 GitLab Runner" } }

The AMI specified (ami-03972092c42e8c0ca) corresponds to Amazon Linux 2, which provides a stable environment for the GitLab Runner service. Using the templatefile function is a critical security and automation practice; it ensures that sensitive tokens are not hardcoded into the configuration but are injected dynamically during the provisioning phase.

Variable Management and terraform.tfvars

To maintain flexibility across different environments (e.g., development, staging, production), variables must be used. The variables.tf file defines the schema, while the terraform.tfvars file provides the actual values.

Essential variables include:

• aws_region: The target AWS region (e.g., us-east-1).
• instance_type: The hardware specification for the EC2 instance (e.g., t2.micro).
• key_name: The name of the SSH key pair existing in the AWS account.
• gitlab_runner_registration_token: The sensitive token retrieved from GitLab settings.

Example terraform.tfvars configuration:

hcl aws_region = "us-east-1" instance_type = "t2.micro" key_name = "your-key-name" gitlab_runner_registration_token = "your-registration-token"

The variable gitlab_runner_registration_token is explicitly marked as sensitive = true. This prevents the token from being printed in plain text within the Terraform terminal output during a terraform plan or terraform apply operation, protecting the integrity of the GitLab project.

Outputting Critical Infrastructure Data

Once the deployment is complete, the user needs immediate access to the instance's identifiers. The output.tf file should be configured to return the Instance ID and the Public IP address.

```hcl
output "instanceid" {
description = "The ID of the instance."
value = awsinstance.gitlab_runner.id
}

output "instancepublicip" {
description = "The public IP of the instance."
value = awsinstance.gitlabrunner.public_ip
}
```

Automated Provisioning via User Data Scripts

The install_runner.tpl file is a shell script that executes automatically when the EC2 instance boots for the first time. This is the "Bootstrap" phase where the generic Amazon Linux image is transformed into a functional GitLab Runner.

The script follows these logical steps:

1. Enables debug mode using set -x to ensure all executed commands are logged to the terminal/cloud-init logs.
2. Updates the local package repository using yum update -y.
3. Installs core utilities like curl and git.
4. Downloads and installs the GitLab Runner binary.

The template structure allows for the dynamic injection of the registration token:

```bash

!/bin/bash

set -x
echo "Hello from EC2 user data script"
yum update -y
yum install -y curl git

Install GitLab Runner

[Installation commands follow...]

```

The use of set -x is vital for troubleshooting. If the registration fails, administrators can inspect the system logs to identify exactly which command caused the failure, reducing the Mean Time to Recovery (MTTR).

Integration with GitLab CI/CD and Remote Backend Management

When running Terraform within a GitLab pipeline, the configuration must be aware of the GitLab-managed remote backend. This prevents "state locking" issues and ensures that multiple developers can collaborate on the same infrastructure without overwriting each other's changes.

Configuring the HTTP Backend

To use GitLab as the backend, the provider.tf file must contain a backend "http" block. This block requires three specific attributes to manage the state and its locking mechanism:

• address: The URL to access the state information.
• lock_address: The URL used to lock the state file during an operation.
• unlock_address: The URL used to unlock the state file once the operation completes.

The configuration template for the backend is as follows:

```hcl
terraform {
requiredproviders {
aws = {
source = "hashicorp/aws"
version = "~> 4.18.0"
}
}
backend "http" {
address = "https://gitlab.com/api/v4/projects//terraform/state/default"
lockaddress = "https://gitlab.com/api/v4/projects//terraform/state/default/lock"
unlock_address = "https://gitlab.com/api/v4/projects//terraform/state/default/lock"
}
}

provider "aws" {
region = "eu-central-1"
}
```

Users can obtain these exact URLs by navigating to the GitLab interface under "Operate > Terraform states" (formerly "Infrastructure > Terraform") and clicking "Copy Terraform init command."

Handling Authentication Failures

A common hurdle in this automation is the lack of AWS credentials during the initial pipeline run. When the pipeline triggers, the Terraform AWS Provider attempts to authenticate with AWS and fails because no keys are present in the environment.

To resolve this, the following steps are mandatory:

1. Navigate to the GitLab project.
2. Go to "Settings > CI/CD".
3. Expand the "Variables" section.
4. Add AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

By adding these as CI/CD variables, GitLab injects them into the runner's environment as environment variables. Terraform's AWS provider is designed to automatically look for these specific variable names, allowing for seamless, passwordless authentication within the pipeline.

Verification and Operational Maintenance

Once terraform apply has successfully completed, the infrastructure must be validated through both the AWS console and the GitLab interface.

Manual Verification via SSH

To ensure the service is running correctly on the host, establish an SSH connection:

bash ssh -i "your-key-name.pem" ec2-user@your-instance-public-ip

Once connected, use systemctl to verify the service status:

bash systemctl status gitlab-runner.service

If the service is active, the runner is operational. On the GitLab web interface, navigate to "Project > Setting > CICD" and expand the "Runners" section. The new EC2 instance should appear as an active, online runner.

Pipeline Testing

To confirm the runner can actually execute jobs, a .gitlab-ci.yml file must be added to the root of the repository. This file defines a test stage to verify the runner's environment.

```yaml

.gitlab-ci.yml

stages:
- verify-runner

verify-runner:
stage: verify-runner
script:
- echo "Hello, World!"
- cat /etc/os-release
- hostname -f
- date
tags:
- aws
```

The tags attribute is critical. It tells GitLab to only send this job to runners that have the aws tag, which matches the tag assigned during the Terraform/User Data registration process.

Resource Decommissioning

One of the primary benefits of using Terraform is the ability to cleanly decommission resources. To avoid unnecessary AWS costs associated with idle EC2 instances, use the destroy command:

bash terraform destroy

This command will read the state file and systematically remove the EC2 instance and the Security Group in the correct dependency order.

Advanced Scaling with GitLab Runner Infrastructure Toolkit (GRIT)

For organizations requiring higher levels of complexity, such as autoscaling environments, the GitLab Runner Infrastructure Toolkit (GRIT) offers a more advanced approach. GRIT is a library of Terraform modules designed to deploy sophisticated runner configurations on public clouds.

Implementing an Autoscaling Linux Docker Runner

Using GRIT, a user can deploy a runner that utilizes the docker-autoscaler executor. This executor does not rely on a single static instance but instead manages an AWS Auto Scaling Group (ASG) that expands and contracts based on the current job workload.

The implementation requires setting the following variables:

• GITLAB_TOKEN
• AWS_REGION
• AWS_SECRET_ACCESS_KEY
• AWS_ACCESS_KEY_ID

The Terraform module configuration for a GRIT-based autoscaler is as follows:

hcl module "runner" { source = ".local/grit/scenarios/aws/linux/docker-autoscaler-default" name = "grit-runner" gitlab_project_id = "39258790" runner_description = "Autoscaling Linux Docker runner on AWS deployed with GRIT." runner_tags = ["aws", "linux"] max_instances = 5 min_support = "experimental" }

This configuration allows for a maximum of 5 instances to be provisioned. The runner manager utilizes a new VPC and an ASG that uses a public AMI owned by the runner team. This provides a highly elastic execution environment that optimizes both performance and cost.

Analysis of Orchestration Methodologies

The transition from manual runner setup to Terraform-based automation represents a significant leap in operational maturity. The methodologies discussed—ranging from basic EC2 provisioning to the sophisticated GRIT autoscaling modules—provide a spectrum of capabilities suited for different organizational needs.

The fundamental principle across all methods is the decoupling of the runner logic from the underlying infrastructure. By treating the runner as a transient resource that can be defined, deployed, and destroyed through code, teams eliminate the "snowflake server" problem where individual runners become unique and difficult to manage. The use of GitLab's remote HTTP backend further cements this by ensuring that the "state" of the world is always synchronized across the entire DevOps team.

Ultimately, the success of this implementation depends on the rigorous application of security best practices, such as the use of sensitive variables for tokens and the implementation of the principle of least privilege via Security Groups. Whether using a simple t2.micro instance for small projects or a complex GRIT-managed ASG for enterprise workloads, the integration of Terraform and AWS provides a scalable foundation for modern software delivery.

Sources

1. Automating GitLab Runner Setup on AWS EC2 using Terraform
2. GitLab Terraform Integration
3. GitLab Runner Documentation