Terraform organizations that operate on Google Cloud Platform can use Google Cloud Storage as a private distribution point for modules and as a durable backend for state. GCS provides a straightforward mechanism for hosting module archives and state objects with IAM-controlled access, object versioning for recovery, and lifecycle management for cost control. The workflow covers bucket creation with versioning and uniform access, packaging modules as zip archives, uploading with versioned paths, referencing modules with the gcs:: source prefix, and configuring a backend "gcs" for remote state with locking and remote state data sources.
How GCS Module Sources Work
Terraform downloads modules from GCS when you use the gcs:: source prefix. The module is packaged as a zip file, uploaded to a GCS bucket, and referenced using the gcs:: prefix. Google Cloud IAM handles access control, and GCS object versioning gives you version history for overwritten or deleted module archives.
The guide walks through the entire workflow, from setting up the bucket to referencing modules in Terraform configurations.
A complete working example uses a versions.tf with required providers and a GCS backend for state, and a main.tf that references network and GKE modules from GCS.
versions.tf
hcl
terraform {
required_version = ">= 1.5.0"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.0"
}
}
backend "gcs" {
bucket = "myorg-terraform-state"
prefix = "prod/infrastructure"
}
}
main.tf
```hcl
provider "google" {
project = var.project_id
region = var.region
}
module "network" {
source = "gcs::https://www.googleapis.com/storage/v1/myorg-terraform-modules/network/v2.1.0.zip"
projectid = var.projectid
networkname = "${var.environment}-vpc"
region = var.region
subnets = [
{
name = "app-subnet"
ipcidrrange = "10.0.1.0/24"
region = var.region
},
{
name = "data-subnet"
ipcidr_range = "10.0.2.0/24"
region = var.region
},
]
}
module "gke" {
source = "gcs::https://www.googleapis.com/storage/v1/myorg-terraform-modules/gke-cluster/v1.5.0.zip"
projectid = var.projectid
clustername = "${var.environment}-gke"
region = var.region
network = module.network.networkname
subnetwork = module.network.subnetnames[0]
nodecount = var.environment == "prod"
}
```
Compared to Git, you lose code review integration for module changes but gain simpler distribution. GCS provides a straightforward way to host private Terraform modules for GCP-focused organizations. Package modules as zip files, upload to a GCS bucket with a version-based path, and reference them with the gcs:: prefix. Authentication flows through the standard Google Cloud credential chain, and IAM policies control who can download modules.
Bucket Provisioning for Module Hosting
The bucket for storing Terraform modules can be provisioned with Terraform itself.
```hcl
resource "googlestoragebucket" "modules" {
name = "myorg-terraform-modules"
location = "US"
project = var.projectid
forcedestroy = false
versioning {
enabled = true
}
uniformbucketlevel_access = true
lifecyclerule {
condition {
numnewerversions = 5
withstate = "ARCHIVED"
}
action {
type = "Delete"
}
}
labels = {
purpose = "terraform-modules"
managed_by = "terraform"
}
}
```
Or create it with gcloud:
```bash
gcloud storage buckets create gs://myorg-terraform-modules \
--location=US \
--uniform-bucket-level-access \
--project=my-project
gcloud storage buckets update gs://myorg-terraform-modules \
--versioning
```
Organization of the bucket with a clear version-based path is recommended. Example paths are gs://myorg-terraform-modules/network/v1.0.0.zip and gs://myorg-terraform-modules/network/v1.1.0.zip.
A publish script can automate packaging and upload:
bash
echo "Publishing ${MODULE_NAME} ${VERSION}..."
gcloud storage cp "$ZIP_FILE" "gs://${BUCKET}/${MODULE_NAME}/${VERSION}.zip"
echo "Published ${MODULE_NAME} ${VERSION} successfully."
echo "Source: gcs::https://www.googleapis.com/storage/v1/${BUCKET}/${MODULE_NAME}/${VERSION}.zip"
Upload commands:
bash
gcloud storage cp /tmp/network-v1.0.0.zip gs://myorg-terraform-modules/network/v1.0.0.zip
gcloud storage cp /tmp/network-v1.1.0.zip gs://myorg-terraform-modules/network/v1.1.0.zip
gcloud storage ls gs://myorg-terraform-modules/network/
The bucket configuration supports lifecycle rules to clean up old versions, uniform bucket-level access for consistent IAM, and versioning for module archive history.
Bucket Configuration Specs
| Attribute | Example Value | Purpose |
|---|---|---|
| name | myorg-terraform-modules | Bucket identifier for module hosting |
| location | US | Regional placement |
| versioning.enabled | true | Enable version history for archives |
| uniformbucketlevel_access | true | Recommended uniform access model |
| lifecyclerule.condition.numnewer_versions | 5 | Keep last 5 archived versions |
| lifecycle_rule.action.type | Delete | Action for expired versions |
GCS as Terraform State Backend
GCS stores the state as an object in a configurable prefix in a pre-existing bucket on Google Cloud Storage. The bucket must exist prior to configuring the backend. This backend supports state locking.
It is highly recommended that you enable Object Versioning on the GCS bucket to allow for state recovery in the case of accidental deletions and human error.
hcl
terraform {
backend "gcs" {
bucket = "tf-state-prod"
prefix = "terraform/state"
}
}
Remote state can be read with a data source:
```hcl
data "terraformremotestate" "foo" {
backend = "gcs"
config = {
bucket = "terraform-state"
prefix = "prod"
}
}
resource "localfile" "foo" {
content = data.terraformremote_state.foo.outputs.greeting
filename = "${path.module}/outputs.txt"
}
```
For Terraform <= 0.11 the interpolation syntax is used:
hcl
resource "local_file" "foo" {
content = "${data.terraform_remote_state.foo.greeting}"
filename = "${path.module}/outputs.txt"
}
IAM Changes to buckets are eventually consistent and may take up to a few minutes to take effect. Terraform will return 403 errors till it is eventually consistent.
If you are using terraform on your workstation, you will need to install the Google Cloud SDK and authenticate using User Application Default Credentials. User ADCs do expire and you can refresh them by running gcloud auth application-default login. If you are running terraform on Google Cloud, you can configure that instance or cluster to use a Google Service Account.
Module Distribution Modules
The terraform-google-modules/cloud-storage module makes it easy to create one or more GCS buckets, and assign basic permissions on them to arbitrary users.
The resources/services/activations/deletions that this module will create/trigger are:
- One or more GCS buckets
- Zero or more IAM bindings for those buckets
If you only wish to create a single bucket, consider using the simple bucket submodule instead. This module is meant for use with Terraform 0.13+ and tested using Terraform 1.0+. If you find incompatibilities using Terraform >=0.13, please open an issue. If you haven't upgraded and need a Terraform 0.12.x-compatible version of this module, the last released version intended for Terraform 0.12.x is v1.7.1.
Basic usage:
hcl
module "gcs_buckets" {
source = "terraform-google-modules/cloud-storage/google"
version = "~> 12.3"
project_id = "<PROJECT ID>"
names = ["first", "second"]
prefix = "my-unique-prefix"
set_admin_roles = true
admins = ["group:[email protected]"]
versioning = {
first = true
}
bucket_admins = {
second = "user:[email protected],user:[email protected]"
}
}
Functional examples are included in the examples directory.
Module Input Reference
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| admins | IAM-style members who will be granted roles/storage.objectAdmin on all buckets. | list(string) | [] | no |
| autoclass | Optional map of lowercase unprefixed bucket name => boolean, defaults to false |
Production-Ready Bucket Management
A production-ready Terraform module for managing Google Cloud Storage buckets with lifecycle rules, versioning, CORS, retention policies, uniform bucket-level access, CMEK encryption, Pub/Sub notifications, IAM bindings, and autoclass.
The module architecture spans Storage Config, Security, Data Management, and Integrations.
Storage Config covers Storage Class, Location, and Autoclass. Security covers Uniform Access, CMEK Encryption, Public Access Prevention, and IAM Bindings. Data Management covers Lifecycle Rules, Versioning, Retention Policy, Soft Delete. Integrations cover Pub/Sub Notifications, Access Logging, Static Website.
Lifecycle Rules support Delete, Archive, Transition actions.
Upload and Versioning Workflow
Upload your packaged module using gcloud or gsutil:
bash
gcloud storage cp /tmp/network-v1.0.0.zip gs://myorg-terraform-modules/network/v1.0.0.zip
gcloud storage cp /tmp/network-v1.1.0.zip gs://myorg-terraform-modules/network/v1.1.0.zip
gcloud storage ls gs://myorg-terraform-modules/network/
Organize your bucket with a clear version-based path. Object versioning gives you version history for overwritten or deleted module archives.
Summary
GCS provides a straightforward way to host private Terraform modules for GCP-focused organizations. Package modules as zip files, upload to a GCS bucket with a version-based path, and reference them with the gcs:: prefix. Authentication flows through the standard Google Cloud credential chain, and IAM policies control who can download modules. With a simple publish script and a consistent naming convention, you have a module distribution system that requires no additional infrastructure beyond what GCP already provides.
Conclusion
Hosting Terraform modules and state in GCS combines native GCP capabilities with Terraform-native source prefixes and backends. Module hosting relies on a versioned bucket with object versioning enabled, uniform bucket-level access, and lifecycle rules for archival cleanup. Modules are distributed as zip files under a clear module-name/version.zip path and consumed via gcs::https://www.googleapis.com/storage/v1/... sources. State backends use a pre-existing bucket with a configurable prefix, support state locking, and benefit from object versioning for recovery. IAM propagation is eventually consistent, so authentication must be refreshed for workstations using User Application Default Credentials or service accounts for Google Cloud runs. Community modules such as terraform-google-modules/cloud-storage provide bulk bucket creation and IAM binding, while production modules add CMEK encryption, CORS, retention, Pub/Sub notifications, and autoclass. The result is a private, GCP-native module distribution and state storage pattern without additional infrastructure.