GitLab CI/CD Infrastructure Automation with Terraform and OpenTofu

The integration of Infrastructure as Code (IaC) into a continuous delivery pipeline represents a fundamental shift in how modern cloud environments are managed. By utilizing GitLab, organizations can move away from manual infrastructure provisioning and embrace a declarative approach where the state of the environment is defined in version-controlled files. This synergy allows teams to treat their infrastructure with the same rigor as their application code, applying the same testing, validation, and approval workflows to the underlying hardware and network configurations.

The core of this automation lies in the GitLab CI/CD pipeline, which orchestrates the transition from a code change in a Git repository to a deployed resource in a cloud provider. When a developer pushes a change to a Terraform configuration, the pipeline triggers a series of automated stages—typically validate, plan, and apply. This ensures that syntax errors are caught early, the intended changes are visible before they are executed, and the actual deployment is performed in a consistent, repeatable manner.

Historically, GitLab provided a seamless experience through integrated templates, but the landscape of the industry shifted significantly due to HashiCorp's license changes. This transition has led to the deprecation of certain native Terraform templates in GitLab 18.0 and the rise of OpenTofu, an open-source fork of Terraform version 1.5.6. Understanding the current state of these tools and how to navigate the transition between Terraform and OpenTofu is critical for any engineer maintaining a production-grade infrastructure pipeline.

The Architecture of GitLab Terraform Pipelines

GitLab provides a comprehensive ecosystem for managing IaC, distinguishing itself from other platforms like GitHub Actions through its deep native integration of state management and pipeline components. While other platforms often rely on third-party actions and external storage solutions for state, GitLab offers a cohesive, built-in experience.

Native GitLab Integration Advantages

The decision to use GitLab for Terraform workflows is often driven by the need for a single pane of glass for both application and infrastructure code.

• Unified Platform: GitLab allows teams to plan, validate, and apply infrastructure changes in the same environment where the application code resides, reducing context switching.
• Built-in State Backend: GitLab includes a managed HTTP Terraform state backend, eliminating the need for an external S3 bucket or GCS bucket to store state files.
• Merge Request Visibility: The platform provides specific diff views for plan output within merge requests, allowing reviewers to see exactly what resources will be created, modified, or destroyed before approving a change.
• Access Control: GitLab's permission system enforces approval workflows, ensuring that an apply run cannot occur without the necessary administrative sign-off.

The GitLab Terraform HTTP State Backend

One of the most significant technical advantages of using GitLab for IaC is the integrated state management. The Terraform state file is a critical piece of metadata that maps real-world resources to your configuration.

The GitLab-managed state is scoped per project and is protected by the project's existing access controls. This means that only users with the appropriate permissions can access or modify the state file. To implement this, the Terraform backend block in the configuration must be pointed to the GitLab instance URL. Authentication is handled via a personal access token or a CI/CD job token, which is automatically provided by the GitLab runner during the execution of the pipeline. This removes the overhead of managing external cloud storage for state and centralizes the security model.

The Transition from Terraform to OpenTofu

A critical shift occurred in the ecosystem following HashiCorp's change in licensing. This has direct implications for how GitLab delivers its CI/CD templates.

The Deprecation of Official Templates

The official GitLab Terraform CI/CD template, specifically Terraform/Base.gitlab-ci.yml, was deprecated and subsequently fully removed in GitLab 18.0. This means that any pipeline relying on this specific template will fail upon upgrading to the latest version of GitLab.

For organizations that find themselves in this situation, there are three primary paths forward:

• Pinned Releases: Users can pin their pipeline to a specific older GitLab release, such as ref: 'v17.0.0-ee', to maintain the use of the legacy Terraform-based template. This is a temporary fix and prevents the project from benefiting from newer GitLab features.
• Migration to OpenTofu: The officially recommended path is to migrate to the GitLab OpenTofu CI/CD component. OpenTofu is an open-source alternative forked from Terraform 1.5.6 and is designed to be a viable, compatible replacement.
• Self-Hosting Images: GitLab maintains a separate repository called gitlab-org/terraform-images. Organizations that wish to continue using Terraform without switching to OpenTofu can fork this repository and self-host the images to ensure their pipelines continue to function.

Implementing an OpenTofu Pipeline

For those adopting the recommended path, the OpenTofu CI/CD component provides a streamlined way to add validate, plan, and apply workflows. This is achieved by including the component in the .gitlab-ci.yml file.

The following configuration demonstrates the implementation of an OpenTofu pipeline:

```yaml
include:
- component: gitlab.com/components/opentofu/validate-plan-apply@

inputs:
version:
opentofuversion:
rootdir: terraform/
state_name: production
stages: [validate, build, deploy]
```

This component-based approach allows for better versioning and modularity compared to the old monolithic templates.

Manual Pipeline Construction using .gitlab-ci.yml

For users who prefer full control or are using a custom image, a pipeline can be built from scratch using the .gitlab-ci.yml file located in the project root directory. This file tells GitLab exactly how to execute the infrastructure workflow.

Pipeline Configuration Components

A basic but effective Terraform pipeline requires a specific set of definitions to ensure the environment is clean and the credentials are secure.

The following is a complete example of a manual Terraform pipeline configuration:

```yaml
image:
name: hashicorp/terraform:light
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

before_script:
- rm -rf .terraform
- terraform --version
- mkdir -p ./creds
- echo $SERVICEACCOUNT | base64 -d > ./creds/serviceaccount.json
- terraform init

stages:
- validate
- plan
- apply

validate:
stage: validate
script:
- terraform validate

plan:
stage: plan
script:
- terraform plan -out "planfile"
dependencies:
- validate
artifacts:
paths:
- planfile

apply:
stage: apply
script:
- terraform apply -input=false "planfile"
dependencies:
- plan
when: manual
```

Deep Analysis of the Pipeline Logic

The logic within the .gitlab-ci.yml file is designed to prevent "state drift" and ensure security.

• Image Specification: The use of hashicorp/terraform:light ensures a lightweight execution environment. The entrypoint override is necessary to ensure the shell environment is correctly configured within the GitLab Runner's Docker container.
• The before_script Phase: This section is critical for initialization. The command rm -rf .terraform ensures that no stale cached providers or modules interfere with the current run. The sequence then verifies the version, creates a credentials directory, decodes a base64 encoded service account key (stored as a GitLab CI/CD variable), and executes terraform init to prepare the working directory.
• The Validate Stage: This is the first line of defense. Running terraform validate checks the syntax of the configuration files without needing to connect to the cloud provider, failing the pipeline early if there are typos or structural errors.
• The Plan Stage: The terraform plan -out "planfile" command generates an execution plan. This plan is saved as an artifact. This is a vital security step because it ensures that the exact plan reviewed by the operator is the one executed in the next stage.
• The Apply Stage: The apply job is configured with when: manual. This creates a "human-in-the-loop" gate. The pipeline will pause after the plan is generated, requiring a user to manually click the "Play" button in the GitLab UI to execute the changes.

Execution Flow and Runner Dynamics

GitLab pipelines are executed by "Runners." In the context of the provided configuration, the runners use ephemeral Docker containers. This means every time a job starts, a fresh container is spun up, the before_script is executed, and the container is destroyed after the job completes.

This ephemeral nature ensures that there is no "leakage" between jobs. For instance, the planfile created in the plan stage is passed to the apply stage using GitLab's artifacts mechanism, even though the container that created the file no longer exists.

Comparative Analysis: GitLab CI/CD vs. GitHub Actions

When deciding on a platform for IaC, it is important to understand the structural differences between GitLab and GitHub's offerings.

Setting Up a New Project for IaC

To begin implementing these workflows, a structured setup process is required.

• Project Creation: Start by creating a new project in GitLab. Selecting a "blank project" is the standard approach. The project name will automatically be translated into a project slug for the URL.
• Visibility Settings: While private repositories are standard for infrastructure, making a repository public can simplify initial Git operations such as cloning and pushing during the testing phase.
• Local Environment Setup: Once the project is created, use the "Clone" option to bring the repository to a local machine. This allows the developer to write the Terraform files and the .gitlab-ci.yml configuration before pushing them to the remote server to trigger the first pipeline.

Advanced Workflow Considerations

While the basic validate-plan-apply flow is sufficient for small projects, enterprise environments often require more sophistication.

• Feature Branch Workflow: Instead of pushing directly to the main branch, infrastructure changes should be developed on feature branches. This allows the validate and plan stages to run on the branch, giving the team a chance to review the proposed changes in a Merge Request before they are merged into the main branch and triggered for apply.
• Resource Cleanup: In testing environments, it is critical to remember that automated pipelines can create expensive resources. For example, deploying a GKE cluster via a pipeline will incur charges until the resources are manually or automatically destroyed.
• Module Registry: GitLab can act as a Terraform/OpenTofu Module Registry, allowing teams to share versioned infrastructure modules across different projects within the organization.
• GitLab Provider: For those looking to manage GitLab itself as code, the GitLab Terraform provider allows the management of users, groups, and projects using Terraform.

Conclusion

The integration of Terraform and OpenTofu into GitLab pipelines transforms infrastructure management from a manual, error-prone process into a disciplined software engineering practice. By leveraging the built-in HTTP state backend, teams eliminate the complexity of managing external state storage and benefit from a security model tied directly to GitLab's access controls.

The shift toward OpenTofu represents a pivotal moment in the IaC ecosystem. While the deprecation of native Terraform templates in GitLab 18.0 initially appears as a disruption, it actually steers users toward a more sustainable, open-source path. Whether an organization chooses to pin to older releases, self-host their own images, or migrate to the OpenTofu CI/CD components, the goal remains the same: achieving a fully automated, transparent, and reversible infrastructure deployment process.

The effectiveness of these pipelines relies on the strict implementation of the "plan-then-apply" pattern. By utilizing artifacts to pass the plan file and requiring manual intervention for the apply stage, GitLab provides the necessary safeguards to prevent catastrophic infrastructure failures. As the ecosystem evolves, the ability to treat the data center as code will continue to be a primary competitive advantage for technical teams.

Sources

1. Spacelift Blog - GitLab Terraform
2. Tim Berry - Terraform Pipelines in GitLab
3. GitLab Documentation - Infrastructure as Code