Architecting Google Cloud Infrastructure with Terraform Modules and Blueprints

The transition from manual resource provisioning to Infrastructure-as-Code (IaC) is a critical evolution for any organization scaling its footprint on Google Cloud Platform (GCP). Manual configuration is inherently tedious, time-consuming, and prone to human error. Terraform solves these challenges by allowing engineers to define their infrastructure in declarative configuration files. However, as the environment grows from a few resources to an enterprise-scale organization, raw Terraform code becomes difficult to manage. This is where Terraform modules and blueprints become indispensable.

By leveraging modularity, organizations can transform their GCP infrastructure into a library of reusable building blocks. This approach ensures consistency across environments, reduces code duplication, and accelerates the deployment of complex, secure architectures.

Understanding Terraform Modules in Practice

At its core, a Terraform module is a logical abstraction of Terraform resources. Rather than writing the same blocks of code for every new environment, a module allows a developer to wrap complex logic—such as the creation of a VPC, subnets, and firewall rules—into a single, callable unit with clean inputs and outputs.

Technically, a module is simply a folder containing one or more .tf files. A professional, production-ready module is typically structured to expose three primary components:

  • Inputs: Defined in variables.tf, these allow the user to pass specific values (like project IDs or region names) into the module.
  • Resources: Defined in main.tf (and potentially other files), these contain the actual Google Cloud resource definitions.
  • Outputs: Defined in outputs.tf, these export information (like a VPC ID or an instance IP address) that other modules or configurations can use.

Recommended Repository Structure

For teams managing multiple environments (e.g., development, staging, production), a clean repository layout is vital to avoid configuration drift and dependency hell. A best-practice structure separates the generic logic (modules) from the environment-specific implementation (envs).

The following layout is recommended for real-world engineering teams:

text terraform/ ├── modules/ │ └── vpc/ │ ├── main.tf │ ├── variables.tf │ ├── outputs.tf │ ├── versions.tf │ └── README.md └── envs/ ├── dev/ │ ├── main.tf │ ├── providers.tf │ ├── backend.tf │ └── terraform.tfvars └── prod/ ├── main.tf ├── providers.tf ├── backend.tf └── terraform.tfvars

In this architecture, the goal is to keep the modules generic and the environment folders thin. The environment folders should merely call the modules and provide the necessary parameters via .tfvars files.

Implementing a Reusable GCP VPC Module

A Virtual Private Cloud (VPC) is the foundation of networking in GCP. Implementing this as a module prevents the repetitive writing of network and firewall logic. A robust VPC module should be capable of creating a single VPC network, a variable number of subnets based on a provided list, and optional firewall rules.

Technical Implementation

To ensure stability, the module must specify the required Terraform version and the Google provider version. For instance, using Terraform version 1.5.0 or higher and Google provider version 5.0.0 or higher ensures access to the latest resource features.

Below is a conceptual implementation of a VPC module call within a development environment:

hcl module "vpc" { source = "../../modules/vpc" project_id = var.project_id vpc_name = "dev-vpc" subnets = [ { name = "dev-us-central1-public" region = "us-central1" cidr = "10.10.0.0/24" private_google_access = true }, { name = "dev-us-central1-private" region = "us-central1" cidr = "10.10.1.0/24" private_google_access = true } ] firewall_rules = [ { name = "dev-allow-ssh-from-office" source_ranges = ["203.0.113.10/32"] target_tags = ["ssh"] protocol = "tcp" ports = ["22"] } ] }

Avoiding Common Modularization Mistakes

When building these modules, experts caution against several common pitfalls:

  • Hardcoding: Never hardcode region names, CIDR blocks, environment names, or project IDs within the module. These must always be passed as variables.
  • Over-Modularization (The "Platform Module" Trap): Avoid creating a single, massive "platform module" that attempts to do everything. Instead, decompose the infrastructure into smaller, specialized modules such as modules/vpc, modules/cloud_run, modules/cloud_sql, and modules/iam.
  • IAM Over-reach: Some IAM resources are authoritative and can inadvertently remove existing access. To avoid this, it is recommended to start with *_iam_member resources and expand as needed.

For teams using version control for their modules, pinning versions is a critical safety measure. This is achieved by referencing a specific Git tag in the source URL:

hcl module "vpc" { source = "git::https://github.com/YOUR_ORG/YOUR_REPO.git//modules/vpc?ref=v1.0.0" }

GCP Terraform Blueprints and the Fabric Framework

While modules provide the building blocks, blueprints provide the architectural plan. A Terraform blueprint is a package of deployable, reusable modules and policies that implement and document a specific, opinionated solution. Essentially, if a module is a "brick," a blueprint is a "pre-fabricated room."

The Cloud Foundation Fabric (Fabric FAST)

Setting up a production-ready GCP organization is often a time-consuming process. To accelerate this, the Cloud Foundation Fabric (specifically Fabric FAST) provides an organization-wide landing zone toolkit. Its purpose is to bootstrap real-world cloud foundations quickly by providing a design that includes typical enterprise elements.

The Fabric approach emphasizes two distinct types of modules:
1. Organization-wide landing zone toolkits for bootstrapping.
2. Lean modules that are easily adaptable to specific organizational changes.

Fabric is designed to be cloned as a single unit and then forked into separate owned repositories for production use, or utilized as-is for prototyping.

Design Principles of Fabric Modules

Modules within the Fabric suite adhere to several strict engineering standards to ensure they are production-ready:
- Provider Proximity: Modules stay as close as possible to the underlying provider resources to avoid unnecessary abstraction layers.
- Integrated IAM: They support IAM configuration alongside the creation and modification of resources.
- Resource Multiplicity: They offer the option to create multiple resources where logical (though not for projects).
- Side-effect Free: They are completely free of side-effects, meaning they do not execute external commands.

Comprehensive Blueprint Categorization

Google Cloud provides a wide array of blueprints to cover different operational and technical needs. These are packaged as modules and categorized by their primary function.

Category Blueprint / Module Description
End-to-end, Data analytics ai-notebook Protects confidential data in Vertex AI Workbench notebooks
Data analytics, End-to-end crmint Deploys the marketing analytics application, CRMint
End-to-end, Operations enterprise-application Deploys an enterprise developer platform on Google Cloud
End-to-end, Operations example-foundation Demonstrates composition of CFT modules for a secure foundation
End-to-end fabric Advanced examples designed for prototyping
Developer tools, End-to-end, Security secure-cicd Builds a secure CI/CD pipeline on Google Cloud
End-to-end, Data analytics secured-data-warehouse Deploys a secured BigQuery data warehouse
Data analytics, Security secured-data-warehouse-onprem-ingest Secured warehouse for ingesting encrypted on-prem data
End-to-end vertex-mlops Creates the Vertex AI environment required for MLOps
Networking address Manages Google Cloud IP addresses
Databases alloy-db Creates an AlloyDB for PostgreSQL instance
Data analytics analytics-lakehouse Deploys a Lakehouse Architecture Solution
Compute anthos-vm Creates VMs on Google Distributed Cloud clusters
Developer tools apphub Creates and manages App Hub

Deep Dive into Fabric Module Capabilities

The Fabric suite of modules is designed for rapid composition. It covers almost all core foundational and networking components, with ongoing development in compute, security, and data scenarios.

Foundational Modules

These modules handle the highest levels of the GCP resource hierarchy and organizational governance:
- Billing account management.
- Cloud Identity group configuration.
- Folder and Project creation.
- Service account provisioning.
- Logging bucket configuration.
- Organization-level settings.
- Projects data-source utilities.

Process Factories

Fabric includes "process factories," such as the project factory, which automates the repetitive task of spinning up standardized projects with pre-defined settings.

Networking Modules

The networking suite in Fabric is extensive, covering nearly every connectivity scenario:
- DNS and DNS Response Policy.
- Cloud Endpoints and address reservation.
- NAT and VLAN Attachments.
- Load Balancing: External Application LB, External Passthrough Network LB, External Regional Application Load Balancer, Internal Application LB, Cross-region Internal Application LB, Internal Passthrough Network LB, and Internal Proxy Network LB.
- Firewall policy and VPC firewalls.
- Interconnectivity: IPSec over Interconnect, VPC peering, and dynamic VPN.

Deployment Workflow and Prerequisites

To deploy infrastructure using these modules, a specific set of prerequisites must be met to ensure the environment is secure and compatible.

Prerequisites

  • Google Cloud Account: A fully configured account with an active project.
  • Terraform Installation: The Terraform CLI installed on the local machine or a CI/CD runner.
  • Proper Authentication: Configured credentials to allow Terraform to communicate with the GCP APIs.

The Deployment Lifecycle

  1. Initialization: Run terraform init to download the required providers and modules.
  2. Planning: Use terraform plan to preview the changes. This step is critical to ensure no unexpected resources are deleted.
  3. Application: Execute terraform apply to provision the resources defined in the module.
  4. Validation: Verify that the VPC, subnets, and firewall rules are operational.

Conclusion

The adoption of Terraform modules and blueprints represents a fundamental shift in how Google Cloud environments are managed. By moving away from monolithic scripts and manual clicks, organizations can implement a "Lego-block" approach to infrastructure. This modularity leads to significantly less code duplication, safer change management, and the ability to maintain consistent environments across the entire software development lifecycle.

The Cloud Foundation Fabric (FAST) and the variety of available blueprints provide a shortcut for enterprise-grade deployments. Whether it is deploying a secure CI/CD pipeline, a Vertex AI MLOps environment, or a complex networking topology involving multiple load balancers and VPNs, the combination of a well-structured module library and opinionated blueprints ensures that security and scalability are baked into the infrastructure from day one. For the modern DevOps engineer, the goal is no longer just to "deploy the cloud," but to build a sustainable, reusable platform that can evolve alongside the business requirements without introducing instability.

Sources

  1. cloud-foundation-fabric
  2. terraform-modules-for-reusable-gcp-infrastructure-with-a-real-vpc-module-example-4b
  3. terraform-blueprints
  4. deploying-gcp-infrastructure-using-terraform-modules-a-step-by-step-guide-365n

Related Posts