SOPS-encrypted secrets have become a common pattern for teams that want to keep sensitive configuration values in version control without exposing plaintext. The combination of Mozilla SOPS for encryption at rest and a Terraform provider for decryption at plan and apply time creates a bridge between infrastructure-as-code workflows and secure secret operations. The provider landscape includes distinct implementations that differ in how they expose SOPS data to Terraform, with security requirements that are consistent across variants.
Provider Variants and Architectural Intent
The Terraform ecosystem contains multiple SOPS integration approaches. One implementation is described as a Terraform provider for SOPS providing functions to decrypt SOPS-encrypted files and strings. This provider is heavily inspired by carlpett/terraform-provider-sops but does not rely on data sources and instead uses provider functions.
The terraform-provider-sops project is a Terraform provider that enables secure integration of SOPS-encrypted secrets into Terraform configurations. This document provides a comprehensive overview of the terraform-provider-sops project, explaining its purpose, architecture, and core components. It covers the provider's integration with Mozilla SOPS for secure secret management within Terraform configurations.
The provider acts as a bridge between Terraform's infrastructure-as-code capabilities and SOPS's secret management, allowing teams to store sensitive configuration data encrypted at rest while making it accessible during Terraform execution.
A Terraform plugin for using files encrypted with SOPS is also documented with explicit security notes. NOTE: To prevent plaintext secrets from being written to disk, you must set up a secure remote state backend. See the official docs on Sensitive Data in State for more information or use ephemeral block. NOTE: All examples assume Terraform 0.13 or newer. For information about usage on older versions, see the legacy usage docs.
Security Model and Operational Requirements
Critical Security Requirement: To prevent plaintext secrets from being written to disk, you must configure a secure remote state backend when using this provider.
The provider consists of two main data sources that handle different secret retrieval scenarios. The provider implements a common data processing pipeline shared between both data sources.
Security measures are documented as follows:
| Security Feature | Implementation | Purpose |
|---|---|---|
| In-Memory Decryption | All decryption occurs in memory only | Prevents plaintext secrets from being written to disk |
| Sensitive Attributes | Terraform marks outputs as sensitive | Prevents accidental exposure in logs and console output |
| Remote State Requirement | Must use secure remote state backend | Ensures encrypted storage of Terraform state containing secrets |
| Multi-Key Support | Supports multiple encryption backends | Enables key rotation and hybrid key management strategies |
The provider implements several security measures to protect sensitive data. The security architecture is built around in-memory handling and state protection.
Security Implementation Model
The provider supports multiple data formats and encryption backends to accommodate different use cases.
Data Sources and Usage Patterns
The provider supports two primary usage patterns:
sops_file
Used when encrypted files are stored locally in the Terraform working directory:sops_external
Used when encrypted content comes from external sources (HTTP endpoints, other Terraform providers):
Key Differences
| Attribute | sops_file | sops_external |
|---|---|---|
| Format detection | auto-detects format from file extension | requires explicit input_type |
| Input source | reads from local filesystem | accepts string content |
| Output types | data (structured map) and raw (decrypted string) | data (structured map) and raw (decrypted string) |
The provider implements a common data processing pipeline shared between both data sources.
Core Provider Components
The provider implements a common data processing pipeline shared between both data sources.
A typical Terraform configuration using the carlpett provider is:
terraform {
required_providers {
sops = {
source = "carlpett/sops"
version = "~> 0.5"
}
}
}
Data source definition:
data "sops_file" "demo-secret" {
source_file = "demo-secret.enc.json"
}
Outputs from decrypted data:
output "root-value-password" {
value = data.sops_file.demo-secret.data["password"]
}
output "mapped-nested-value" {
value = data.sops_file.demo-secret.data["db.password"]
}
output "nested-json-value" {
value = jsondecode(data.sops_file.demo-secret.raw).db.password
}
Sops also supports encrypting the entire file when in other formats.
Encrypt a file using Sops:
sops demo-secret.enc.json
{
"password": "foo",
"db": {"password": "bar"}
}
Encryption Backends and File Formats
SOPS stands for Secrets OPerationS, and is an open-source text file editor that encrypts/decrypts YAML, JSON, ENV, INI and BINARY formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP.
SOPS (Secrets OPerationS) is Mozilla's editor for encrypted files that supports multiple encryption backends and data formats.
Supported Data Formats
- json
- JSON structured data
- yaml
- YAML structured data
- ini
- INI configuration files
- dotenv
- Environment variable files (.env)
- raw
- Raw encrypted content (entire file encrypted)
The provider supports multiple data formats and encryption backends to accommodate different use cases.
SOPS only encrypts the secrets, so the advantage is you can have key-value pairs in a yaml or json file, and while the key is readable, the value part is encrypted. This also allows to address particular keys.
Azure Key Vault Integration Workflow
We are heavily using Terraform and and also Azure. However until now, we left out certain things cause they contain secrets which we don’t want to expose in the code. SOPS is a nice solution to solve that problem and keep things together what belongs together.
If you want more details to SOPS as such you can also have a look at A Comprehensive Guide to SOPS: Managing Your Secrets Like A Visionary, Not a Functionary.
As we will use azure-key-vault: you will need
access to the respective key-vault
sops binary
Encrypting/decrypting with Azure Key Vault requires the resource identifier for a key. This has the following form:
We setup a keyvault (Key Vault names are globally unique) - you can do this using azure cli (as per example) or prefferable using terraform
We create a key - you can do this using azure cli (as per example) or prefferable using terraform
Once we have the key created and configured, then we can use it for encryption
Using both of these keys with SOPS, we can create a file that the cloud-team (owner of the application) is able to edit, and the workspace(s) are able to use for the Terraform configuration. This file is checked into the version control system (GitHub in our case). We typically create a file for each our our environments containing our secrets with the naming-convention secrets.{environment}.enc.yaml
- secrets.dev.enc.yaml
- secrets.test.enc.yaml
- secrets.stage.enc.yaml
- secrets.prod.enc.yaml
To create these files you need to obtain the different keys mentioned above before running the following command:
sops --azure-kv "$WORKSPACE_KV_KEY" --age "$WORKSPACE_KEY" "$FILENAME"
Here WORKSPACEKVKEY
is the Azure key vault key and WORKSPACE_KEY
is the public Age key for the Terraform workspace. The last argument is the FILENAME
being the name of the encrypted file we want to create. A prerequisite for running this command is to have a valid Azure session in our shell, obtainable when running the following command from Azure CLI: az login
Multi-Environment Secret Management
Teams adopting SOPS with Terraform often separate secrets per environment while keeping the same encryption backend. The naming convention secrets.{environment}.enc.yaml provides a clear mapping between workspace and secret file.
The workflow allows the cloud-team to edit encrypted files without exposing values, while workspaces decrypt at Terraform runtime using the appropriate Azure Key Vault key and Age key for the workspace. This separation enables key rotation and hybrid key management strategies through Multi-Key Support.
The provider acts as a bridge between Terraform's infrastructure-as-code capabilities and SOPS's secret management, allowing teams to store sensitive configuration data encrypted at rest while making it accessible during Terraform execution.
Practical Terraform Configuration
When using the carlpett provider, the data source sopsfile is the primary entry point for locally stored encrypted files. The sourcefile argument points to the encrypted file in the working directory. The data attribute exposes a structured map of decrypted values, while raw provides the decrypted file content as a string.
For external content, sopsexternal requires explicit inputtype because format cannot be auto-detected from a file extension. This pattern is useful when secret content is fetched from a remote store or generated by another provider.
All decryption occurs in memory only. Terraform marks outputs as sensitive to prevent accidental exposure in logs and console output. A secure remote state backend is required to ensure encrypted storage of Terraform state containing secrets.
Conclusion
The Terraform SOPS provider family provides a consistent model for bringing encrypted secrets into infrastructure pipelines without placing plaintext in code or state. The architecture centers on in-memory decryption, sensitive attribute handling, and a mandatory secure remote state backend to protect decrypted values after plan and apply.
Two data sources cover the main retrieval scenarios: sopsfile for local encrypted files with auto-detected formats, and sopsexternal for content supplied as strings from external sources with explicit type declarations. Supported formats include json, yaml, ini, dotenv, and raw full-file encryption. Supported encryption backends span AWS KMS, GCP KMS, Azure Key Vault, age, and PGP, with Azure Key Vault workflows commonly paired with Age keys for workspace-specific access.
Operational security depends on correct key management, remote state encryption, and avoidance of disk writes for plaintext. The provider functions variant offers an alternative to data sources for teams preferring function-based decryption. With proper setup, SOPS-encrypted files can be checked into version control per environment, edited by application teams, and consumed safely by Terraform workspaces.