Terraform Pub/Sub Topics and Subscriptions as Code

Google Cloud Pub/Sub is one of those services that shows up in almost every GCP architecture. Whether you are building event-driven microservices, streaming data into BigQuery, or triggering Cloud Functions, Pub/Sub is usually the glue in between. And if you are managing your infrastructure with Terraform, you will want to define your topics and subscriptions as code rather than clicking through the Console.

This guide walks through creating Pub/Sub topics and subscriptions with Terraform, covering the common configurations you will need in a real project.

Provider Configuration and Variables

Before creating any Pub/Sub resources, make sure your Terraform configuration includes the Google provider.

hcl terraform { required_providers { google = { source = "hashicorp/google" version = "~> 5.0" } } } provider "google" { project = var.project_id region = var.region }

A common alternative provider constraint used in project templates is:

hcl terraform { required_providers { google = { source = "hashicorp/google" version = "~> 4.0" } } }

Variables for project and region are defined as follows.

hcl variable "project_id" { description = "The ID of the GCP project" type = string } variable "region" { description = "The region to deploy resources to" type = string default = "us-central1" }

Project structure for a reusable Pub/Sub setup can be organized as:

. ├── main.tf ├── variables.tf ├── outputs.tf ├── terraform.tfvars └── modules/ └── pubsub/ ├── main.tf ├── variables.tf ├── topics.tf └── outputs.tf

Prerequisites for working with this setup include Google Cloud SDK installed and configured, Terraform installed version 1.0.0 or later, and a GCP project with billing enabled.

Creating a Basic Topic

The simplest Pub/Sub setup is a topic with a pull subscription.

hcl resource "google_pubsub_topic" "topic" { name = var.topic_name labels = { environment = "production" } message_retention_duration = "86600s" }

Message retention duration is often set to 7 days for production workloads.

hcl resource "google_pubsub_topic" "topics" { for_each = var.pubsub_config name = each.key project = var.project_id message_retention_duration = "604800s" }

The 604800s value equals 7 days. A 24 hour retention example uses 86600s.

Subscription Configuration

Creating subscriptions for every topic can be done with for_each over a local collection derived from configuration.

hcl variable "pubsub_config" { type = map(object({ subscriptions = list(object({ name = string subscribers = list(string) })) publishers = list(string) })) description = "Pub/Sub configuration, use topics name as key" } locals { subscriptions = flatten([ for topic, config in var.pubsub_config : [ for sub in config.subscriptions : { topic = topic name = sub.name subscribers = sub.subscribers } if length(config.subscriptions) > 0 ] ]) }

The subscription resource is then created from the local map.

hcl resource "google_pubsub_subscription" "subscriptions" { for_each = tomap({ for subscription in local.subscriptions : "${subscription.topic}-${subscription.name}-subscription" => { name = subscription.name topic = subscription.topic } }) name = each.value.name topic = google_pubsub_topic.topics[each.value.topic].name project = var.project_id ack_deadline_seconds = 600 }

Variable definitions for a simple single-topic module are:

hcl variable "topic_name" { description = "Name of the Pub/Sub topic" type = string } variable "subscription_name" { description = "Name of the Pub/Sub subscription" type = string }

IAM Permissions for Pub/Sub

Granting roles to service accounts is required for publisher and subscriber patterns.

hcl resource "google_pubsub_topic_iam_member" "dlq_publisher" { topic = google_pubsub_topic.order_events_dlq.name role = "roles/pubsub.publisher" member = "serviceAccount:service-${data.google_project.current.number}@gcp-sa-pubsub.iam.gserviceaccount.com" } resource "google_pubsub_subscription_iam_member" "dlq_subscriber" { subscription = google_pubsub_subscription.order_processor_with_dlq.name role = "roles/pubsub.subscriber" member = "serviceAccount:service-${data.google_project.current.number}@gcp-sa-pubsub.iam.gserviceaccount.com" } data "google_project" "current" {}

A reusable module often defines the service account locally.

hcl data "google_project" "current" {} locals { pubsub_service_account = "serviceAccount:service-${data.google_project.current.number}@gcp-sa-pubsub.iam.gserviceaccount.com" }

The module also creates a topic IAM binding for the publisher role.

hcl resource "google_pubsub_topic_iam_binding"

Dead Letter Queue and Reusable Module Pattern

If you have many topics and subscriptions that follow similar patterns, wrap them in a Terraform module.

```hcl

modules/pubsub-topic/main.tf - Reusable module for topic with DLQ

variable "topicname" {
type = string
}
variable "subscriptions" {
type = map(object({
ack
deadline = number
maxdeliveryattempts = number
}))
}
variable "labels" {
type = map(string)
default = {}
}
data "googleproject" "current" {}
locals {
pubsub
serviceaccount = "serviceAccount:service-${data.googleproject.current.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
}
resource "googlepubsubtopic" "main" {
name = var.topicname
labels = var.labels
message
retentionduration = "604800s"
}
resource "google
pubsub_topic" "dlq"
```

The module makes it easy to create Google Cloud Pub/Sub topic and subscriptions associated with the topic. The module is meant for use with Terraform 0.13+ and tested using Terraform 1.0+. If you find incompatibilities using Terraform >=0.13, please open an issue. If you haven't upgraded and need a Terraform 0.12.x-compatible version of this module, the last released version intended for Terraform 0.12.x is v1.9.0.

This is a simple usage of the module.

Schema Settings and Topic Configuration

Advanced topic configuration can include schema settings.

hcl resource "google_pubsub_topic" "topic" { name = var.topic_name labels = { environment = "production" } message_retention_duration = "86600s" schema_settings { schema =

Component Connections

The configuration parameters are based on the terraform-google-pubsub Terraform module.

Component connections that can be wired to a Pub/Sub topic or subscription include:

Connected component Application updates Background information
Compute Engine instance template The Compute Engine instances can publish to and receive messages from the Pub/Sub topic. The Pub/Sub topic ID is added to the Compute Engine instance template metadata. Instance templates
Service account The service account can manage Pub/Sub topics, and pull messages from subscriptions. The roles/pubsub.editor role is added to the service account. The service account name and email information is added to the Pub/Sub pull subscription. Service account
BigQuery The Pub/Sub subscription can write messages to the BigQuery dataset. The BigQuery dataset information is added to the BigQuery subscription fields. BigQuery subscriptions
Cloud Run The Cloud Run service can receive messages or publish to the Pub/Sub topic. The Pub/Sub topic ID is added to the Cloud Run environment variables. The roles/pubsub.publisher and roles/pubsub.subscriber roles are added to the Cloud Run service account. The Cloud Run service is added to the Pub/Sub push and pull subscription fields. Use Pub/Sub with Cloud Run tutorial
Cloud Storage The Pub/Sub subscription can write messages to the Cloud Storage

Terraform for Pub/Sub Functions

This tutorial demonstrates how to deploy a Pub/Sub function by uploading a function source code zip file to a Cloud Storage bucket, using Terraform to provision the resources. Terraform is an open source tool that lets you provision Google Cloud resources with declarative configuration files.

This tutorial uses a Node.js function as an example, but it also works with Python, Go, and Java functions. The instructions are the same regardless of which of these runtimes you are using. See Hashicorp's reference pages for details on using Terraform with the Cloud Functions v2 API.

Objectives include learning how to use Terraform to deploy a Pub/Sub function.

Costs in this document involve billable components of Google Cloud. For details, see Cloud Run functions pricing.

Before you begin, sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project:

  • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
  • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission.

Module for Reusable Patterns

Create a Terraform module to create GCP pubsub resources, including topics, subscriptions and iam permissions.

First define a Terraform variable pubsub_config to store your Pub/Sub configuration, then extract your subscriptions to a local variable.

The Terraform code is placed here.

Define a variable to store pubsub configuration.

hcl variable "pubsub_config" { type = map(object({ subscriptions = list(object({ name = string subscribers = list(string) })) publishers = list(string) })) description = "Pub/Sub configuration, use topics name as key" }

The Terraform module pattern supports creating topics with foreach over var.pubsubconfig and creating subscriptions for every topic.

Conclusion

Defining Pub/Sub topics and subscriptions with Terraform moves Pub/Sub configuration from manual console clicks to versioned, repeatable code. Provider configuration with project and region variables establishes the foundation. Basic topic creation with messageretentionduration controls how long unacknowledged messages persist, with 604800s for seven days and 86600s for approximately 24 hours. Subscription resources built from local derivations enable consistent naming and ackdeadlineseconds tuning. IAM bindings for publisher and subscriber roles, often targeting the Pub/Sub service account derived from the project number, secure access between components. Dead letter queues and reusable modules encapsulate patterns for retries, max delivery attempts, and labels. Schema settings allow schema enforcement when required. Component connection tables clarify how Compute Engine, Service Accounts, BigQuery, Cloud Run, and Cloud Storage integrate with topics and subscriptions. Using a module like terraform-google-pubsub or a custom module built around pubsub_config variables keeps large environments maintainable while preserving the declarative benefits of Terraform for event-driven GCP architectures.

Sources

  1. https://oneuptime.com/blog/post/2026-02-17-how-to-create-pubsub-topics-and-subscriptions-with-terraform/view
  2. https://blog.amyinfo.com/2025-03-15-terraform-pubsub-module/
  3. https://www.thecloudpanda.com/blog/gcp-pubsub-terraform/
  4. https://github.com/terraform-google-modules/terraform-google-pubsub
  5. https://docs.cloud.google.com/application-design-center/docs/configure-pubsub
  6. https://docs.cloud.google.com/functions/docs/tutorials/terraform-pubsub

Related Posts