Introduction
Terraform provides a declarative way to describe and provision Google Cloud resources including Cloud Build. When combined with Google Cloud Build, a serverless CI/CD platform, teams can automate Terraform execution as part of a pipeline. The integration relies on Terraform providers, configuration files, and Cloud Build jobs that execute the Terraform CLI. This article covers how Terraform works with Cloud Build, the available Terraform resources for Cloud Build, practical setup considerations, and the guides and modules that support production use.
How Terraform Works with Google Cloud
HashiCorp Terraform is an infrastructure-as-code tool that lets you provision and manage cloud infrastructure. Terraform provides plugins called providers that let you interact with cloud providers and other APIs. You can use the Terraform provider for Google Cloud to provision and manage Google Cloud resources, including Cloud Build.
This page introduces you to using Terraform with Cloud Build, including an introduction to how Terraform works and some resources to help you get started using Terraform with Google Cloud. You'll also find links to Terraform reference docs for Cloud Build, code examples, and guides for using Terraform to provision Cloud Build resources.
Terraform has a declarative and configuration-oriented syntax, which you can use to describe the infrastructure that you want to provision in your Google Cloud project. After you author this configuration in one or more Terraform configuration files, you can use the Terraform CLI to apply this configuration to your Cloud Build resources.
The following steps explain how Terraform works:
- You describe the infrastructure you want to provision in a Terraform configuration file. You don't need to write code describing how to provision the infrastructure. Terraform provisions the infrastructure for you.
- You run the terraform plan command, which evaluates your configuration and generates an execution plan. You can review the plan and make changes as needed
Each resource block describes one or more infrastructure objects, such as virtual networks or compute instances.
The Terraform CLI workflow that users are familiar with for deploying resources to Google Cloud, or any other cloud provider, is:
bash
terraform init
terraform plan
terraform apply
Manually running these commands is useful for learning Terraform or managing your own personal infrastructure in the Cloud, but in practice you’re going to need to learn how to automate Terraform. Let some robots do it for you. There plenty of ways to do that. Using Cloud Build is just one.
Cloud Build as a Serverless CI/CD Executor for Terraform
Cloud Build is Google Cloud’s serverless CI/CD platform. You give it a yaml configuration with some instructions and it can build, test and deploy your software.
Where to begin with a basic Terraform automation setup?
First you’ll need a directory to work out of (a root Terraform module) with 2 files in it (for now): cloudbuild.yaml and main.tf. You can go and create these yourself, or just clone this branch in this repository that has the files already pre-written for you.
You’ll also need a Google Cloud Project with the Cloud Build API enabled. When you enable the Cloud Build API, your project will generate a default Cloud Build service account that looks like this:
This service account automatically has the Cloud Build Service Account IAM role assigned to it, allowing it to do all the Cloud Build things it needs to be able to do, like manage Google Cloud Storage buckets and access Artifact Registry. You can see a full list of permissions right here.
Tell Terraform what to do. Terraform gets its instructions from .tf files.
It will automatically look for a cloudbuild.yaml file and create a Cloud Build job that will execute the provided instructions. You can see the progress as output in your terminal or head over the Cloud Build History page where you’ll see your job’s status and output:
And there you have it! Cloud Build executing Terraform! That’s all there is to it!
Practical Considerations for Running Terraform in Cloud Build
Wait a second, there must be more to it. While we did indeed run Terraform with Cloud Build we didn’t actually deploy any infrastructure (or anything useful for that matter). There are some things that we need to consider.
Permissions
Cloud Build creates a default service account with those default Cloud Build service account permissions assigned to it. Any additional permissions that this service accounts needs will depend on the cloud resources that you want your Terraform to create. If you’re managing resources across lots of different Google Cloud services, then the Cloud Build service account will inevitably accumulate excessive permissions, resulting in a potential security risk. Yuck.
Terraform state file
You’ll notice that the Terraform state file is no where to be found. The state file that would have been generated to your local disk had you ran terraform apply locally only existed temporarily in the container that Cloud Build ran the command from
CI/CD can get pretty complicated. There are a lot of moving pieces and a lot of automation. But as they say, the only way to eat an elephant is one bite at a time. So here’s a first nibble.
How basic are we talking here? We’re going to use Cloud Build to run Terraform for us in our Google Cloud Project. And when I say a very basic example, I mean a very basic example. This won’t be anything you’ll use in production, but this will be the start of a series of posts that build towards a more intricate solution, discussing some important concepts and considerations along the way.
Terraform Resources Available for Cloud Build
The following table lists the Terraform resources available for Cloud Build:
| Cloud Build service | Terraform Resources | Data sources |
|---|---|---|
| Cloud Build v1 | googlecloudbuildtrigger | |
| Cloud Build v2 | googlecloudbuildv2connectioniampolicy |
Terraform-Based Guides for Cloud Build
The following table lists Terraform-based how-to guides and tutorials for Cloud Build:
| Guide | Details |
|---|---|
| Connect to a GitHub repository | This guide explains how to connect a GitHub repository to Cloud Build using Terraform. |
| Connect to a GitHub Enterprise host | This guide explains how to connect a GitHub Enterprise host to Cloud Build using Terraform. |
| Connect to a GitHub Enterprise repository | This guide explains how to connect a GitHub Enterprise repository to Cloud Build using Terraform. |
| Connect to a GitLab Enterprise Edition host | This guide explains how to connect a GitLab Enterprise Edition host to Cloud Build using Terraform. |
| Connect to a GitLab Enterprise Edition repository | This guide explains how to connect a GitLab Enterprise Edition repository to Cloud Build using Terraform. |
| Connect to a Bitbucket Data Center host | This guide explains how to connect a Bitbucket Data Center host to Cloud Build using Terraform |
Modules and Blueprints for Cloud Build
Modules and blueprints help you automate provisioning and managing of Google Cloud resources at scale. A module is a reusable set of Terraform configuration files that creates a logical abstraction of Terraform resources. A blueprint is a package of deployable and reusable modules, and a policy that implements and documents a specific solution.
The following table lists modules and blueprints related to Cloud Build:
| Module or blueprint | Details |
|---|---|
| Secure CI/CD pipeline | This module enables Google Cloud customers to quickly deploy a secure CI/CD pipeline, implementing many of the functions outlined in Shifting left on security. |
| terraform-google-bootstrap | This module helps bootstrap a Google Cloud organization, creating all the required resources and permissions to start using the Cloud Foundation Toolkit (CFT). For users who want to use Cloud Build and Cloud Source Repositories for foundations code, this module contains a submodule that bootstraps all the required resources. |
Getting Started with Terraform Tutorials
The HashiCorp Terraform learning surface provides entry points for users.
Overview
Get Started
Build, change, and destroy infrastructure with Terraform. Start here to learn the basics of Terraform with your favorite cloud provider.
New Tutorials
Try the newest tutorials for common Terraform tasks and use cases.
Get Certified
Prepare for Terraform certifications with our prep materials.
Build Providers
Use the Terraform Plugin Framework to build providers that use common Go conventions.
What's Next for Terraform and Cloud Build
What's next
- Terraform code samples for Cloud Build
- Terraform on Google Cloud documentation
- Google Cloud provider documentation in HashiCorp
- Infrastructure as code for Google Cloud
Conclusion
Combining Terraform with Google Cloud Build creates a path from declarative infrastructure definitions to automated execution. Terraform’s declarative configuration and provider model lets you describe Cloud Build resources and related Google Cloud objects without writing imperative provisioning code. The Terraform CLI workflow of init, plan, and apply maps naturally to a Cloud Build pipeline that can be triggered by source changes.
A minimal setup centers on a root Terraform module with cloudbuild.yaml and main.tf, and a Google Cloud Project with the Cloud Build API enabled. Enabling the API creates a default Cloud Build service account in the form [email protected] with the Cloud Build Service Account IAM role pre-assigned. That account provides baseline permissions for managing Cloud Storage buckets and accessing Artifact Registry, but any Terraform-driven resources beyond Cloud Build require additional IAM grants to the service account.
The available Terraform resources for Cloud Build are currently limited to googlecloudbuildtrigger for Cloud Build v1 and googlecloudbuildv2connectioniampolicy for Cloud Build v2. Guide coverage focuses on connecting external source hosts to Cloud Build via Terraform, including GitHub repository, GitHub Enterprise host, GitHub Enterprise repository, GitLab Enterprise Edition host, GitLab Enterprise Edition repository, and Bitbucket Data Center host connections.
Production use benefits from reusable modules and blueprints. The Secure CI/CD pipeline module helps deploy a secure pipeline aligned with shifting left on security practices. The terraform-google-bootstrap module helps bootstrap a Google Cloud organization for the Cloud Foundation Toolkit, with a submodule specifically for bootstrapping Cloud Build and Cloud Source Repositories for foundations code.
Practical adoption must address permissions sprawl and state management. The default Cloud Build service account accumulates permissions as Terraform creates resources across services, creating a security risk if not scoped. Terraform state generated inside a Cloud Build container is ephemeral unless explicitly persisted to remote state storage, because the state file only existed temporarily in the container that Cloud Build ran the command from.
Using Cloud Build to run Terraform is a very basic example of automation and not production-ready on its own. It provides the first nibble toward a more intricate solution that must consider least-privilege service accounts, durable remote state, and secure pipeline design.