Terraform Cloud Scheduler Jobs for Google Cloud

Creating scheduled workloads in Google Cloud without managing VMs or custom cron daemons is a core operational pattern. Cloud Scheduler provides a fully managed cron service that can trigger HTTP endpoints, Pub/Sub topics, App Engine tasks, and Batch API calls on a recurring schedule. Terraform makes those schedules declarative, version controlled, and repeatable.

This article covers how to provision Cloud Scheduler with Terraform, covering provider configuration, project structure, target types, retry policies, authentication, and integration patterns such as triggering Batch jobs.

Introduction

Cloud Scheduler has a free tier and running a quickstart should not result in costs. Terraform is an infrastructure as code tool that lets you predictably create, change, and improve cloud infrastructure by using code. You can learn more about using Terraform to provision infrastructure on Google Cloud.

In a typical quickstart you use Terraform to create a cron job for Cloud Scheduler, set a recurring schedule for the job, specify a Pub/Sub topic as the job target, run the job, and verify that the job has run successfully.

Security constraints defined by your organization might prevent completion of the setup steps.

Provider Setup and Prerequisites

Before creating jobs, the Google provider must be configured and prerequisites satisfied.

Prerequisites include Google Cloud SDK installed and configured, Terraform installed version 1.0.0 or later, and a GCP project with billing enabled.

Provider configuration is common across examples.

hcl terraform { required_providers { google = { source = "hashicorp/google" version = "~> 4.0" } } } provider "google" { project = var.project_id region = var.region }

A variant using version ~> 5.0 is also used:

hcl terraform { required_providers { google = { source = "hashicorp/google" version = "~> 5.0" } } } provider "google" { project = var.project_id region = var.region }

Variables are defined for project and region.

hcl variable "project_id" { description = "The ID of the GCP project" type = string } variable "region" { description = "The region to deploy resources to" type = string default = "us-central1" }

Project structure for a modular approach can be:

• main.tf # Main Terraform configuration file
• variables.tf # Variable definitions
• outputs.tf # Output definitions
• terraform.tfvars # Variable values
• modules/
  • scheduler/
    • main.tf # Cloud Scheduler specific configurations
    • variables.tf # Module variables
    • jobs.tf # Job configurations
    • outputs.tf # Module outputs

Creating a Cloud Scheduler Cron Job with Terraform

A quickstart shows how to use Terraform to create a Cloud Scheduler cron job.

Deploy your Terraform resources to create the cron job.

bash terraform init terraform plan terraform apply

At the Enter a value prompt, type yes to proceed with the creation of resources.

Confirm that a job is created:

bash gcloud scheduler jobs describe test-job --location=us-east1

The output should be similar to:

description: test job lastAttemptTime: '2024-04-04T13:56:00.669530Z' name: projects/PROJECT_ID/locations/us-east1/jobs/test-job pubsubTarget: data: dGVzdA== topicName: projects/PROJECT_ID/topics/pubsub_topic schedule: '30 16 * * 7' scheduleTime: '2024-04-04T13:58:00.737907Z' state: ENABLED

You have created a job that sends a message to a Pub/Sub topic at 16:30 on Sundays.

For details, see the Terraform Registry argument reference.

Pub/Sub Target Configuration

Pub/Sub is a common target for decoupled scheduling.

```hcl
resource "googlecloudschedulerjob" "pubsubjob" {
name = "test-job"
description = "test job"
schedule = "30 16 * * 7"
timezone = "America/NewYork"
region = "us-east1"

pubsubtarget {
topicname = "projects/PROJECTID/topics/pubsubtopic"
data = base64encode("test")
}
}
```

HTTP Target Configuration

The most common pattern is triggering an HTTP endpoint on a schedule.

A service account is often created for authentication.

hcl resource "google_service_account" "scheduler" { account_id = "cloud-scheduler-sa" display_name = "Cloud Scheduler Service Account" }

HTTP job example with retry configuration:

```hcl
resource "googlecloudschedulerjob" "hourlyreport" {
name = "hourly-report-job"
description = "Triggers the report generation endpoint every hour"
schedule = "0 * * * *"
timezone = "America/NewYork"
region = var.region

retryconfig {
retrycount = 3
maxretryduration = "300s"
minbackoffduration = "5s"
maxmaxbackoffduration = "60s"
maxdoublings = 3
}

httptarget {
httpmethod = "POST"
uri = "https://your-service-abc123-uc.a.run.app/api/reports/generate"
body = ...
}
}
```

A simpler HTTP job:

```hcl
resource "googlecloudschedulerjob" "httpjob" {
name = "http-job"
description = "HTTP job to trigger an endpoint"
schedule = "*/10 * * * *"
timezone = "America/NewYork"
attempt_deadline = "320s"

http_target {
uri = "..."
}
}
```

Authentication and selection steps for deployment:

bash gcloud auth application-default login gcloud config set project your-project-id

Configure variables in terraform.tfvars:

hcl project_id = "your-project-id" region = "us-central1" job_name = "my-scheduler-job" schedule = "0 */3 * * *" target_uri = "https://your-cloud-run-service-uc.a.run.app"

Deploy:

bash terraform init terraform apply

Cron Schedule Format

Cloud Scheduler uses standard unix-cron format with 5 fields.

Schedule is configurable. Default examples include every 3 hours 0 */3 * * *, every 10 minutes */10 * * * *, every hour 0 * * * *.

Time zone is configurable with default UTC.

Retry Configuration and Reliability

Retry configuration ensures jobs eventually complete even when transient failures occur.

Retry settings typically include:

• retry_count
• maxretryduration
• minbackoffduration
• maxbackoffduration
• max_doublings

The three target types HTTP, Pub/Sub, App Engine cover most scheduling needs.

Scheduling Batch Jobs with Cloud Scheduler

Terraform is an open-source tool that lets you provision and manage infrastructure by specifying the desired state in configuration files. These files can be treated as code and stored in version control systems like GitHub.

Although Terraform doesn't have resources for Batch, you can use Terraform to create Batch jobs. Specifically, you can use Terraform to schedule and run a Cloud Scheduler cron job that targets the Batch API to create and run Batch jobs. Cloud Scheduler is a Google Cloud service that allows you to automatically schedule cron jobs and supports Terraform.

Objectives for this pattern are:

• Create a Terraform directory and a configuration file that defines a Cloud Scheduler cron job that creates Batch jobs.
• Deploy the Terraform configuration to run the cron job.
• Verify that the cron job creates Batch jobs.
• Update the Terraform configuration to pause the cron job so that it stops creating Batch jobs.

Costs in this document use billable components of Google Cloud. When you finish the tasks, you can avoid continued billing by deleting the resources that you created.

Think of Cloud Scheduler as a reliable, managed alternative to running cron on a VM - except you do not have to worry about the VM going down and missing a scheduled run.

By defining scheduled jobs as Terraform code, you get version control, code review, and reproducibility for scheduled workloads.

Deployment Workflow

Typical workflow is:

• Authenticate and select project
• Configure variables
• Initialize Terraform
• Plan changes
• Apply

Outputs after apply include Job Name, Schedule, and Target.

Sequence is User -> Terraform -> GCP Scheduler API with job provisioned and outputs returned.

For processing work that Cloud Scheduler triggers, creating Cloud Tasks queues with Terraform is a common next step.

Target Types Comparison

Conclusion

Terraform Cloud Scheduler integration provides a production ready way to schedule Google Cloud workloads without operational overhead. Provider configuration with project and region variables establishes the foundation. Job resources allow declarative definition of schedule, time zone, retry policy, and target specifics for HTTP, Pub/Sub, and App Engine.

Practical patterns include using service accounts for HTTP authentication, configuring retry policies with exponential backoff, and using Pub/Sub targets for event driven pipelines. The cron format remains standard unix-cron with five fields.

Advanced integration uses Cloud Scheduler to trigger Batch API calls, enabling Terraform managed batch processing even though Terraform lacks native Batch resources. This extends IaC coverage to ephemeral compute.

Version control of scheduler definitions enables auditability, code review, and reproducible deployments across environments. With proper variable management via terraform.tfvars and modular project structure, Cloud Scheduler jobs can be scaled and maintained alongside the rest of the infrastructure.

Sources

1. docs.cloud.google.com
2. thecloudpanda.com
3. docs.cloud.google.com
4. oneuptime.com
5. github.com