AWS SSM Parameters and Cross Account Patching Controlled by Terraform

Managing AWS Systems Manager resources through Terraform creates a tension between declarative infrastructure as code and operational workflows that mutate resources outside of Terraform. Systems Manager Parameter Store holds configuration data and secrets that are often written by CI pipelines, deployment scripts, or manual operators. Systems Manager also provides patch automation that can be centralized from a master account to dozens of child accounts. Terraform can create the resources, enforce tagging, provision IAM roles, and maintain associations, while lifecycle meta arguments and module wrappers allow values to drift safely when external processes update them. The following discussion extracts the concrete patterns described in the reference materials and expands each pattern to its operational impact and architectural context.

The challenge with SSM Parameters under Terraform is drift reversion. A parameter created by Terraform with an initial value will be reverted to that initial value on every subsequent terraform apply if an external process updates the parameter in the console or via the AWS CLI. The parameter remains tracked, unlike creation directly in the console, which leaves the resource untracked. The remediation relies on Terraform’s lifecycle meta argument, specifically the ability to ignore changes to the value attribute. When a parameter is meant to be set once by Terraform and subsequently maintained by another system, ignoring value changes prevents Terraform from overwriting the external update while still managing the parameter’s existence, name, type, tier, and tags.

The lifecycle behavior is visible in a simple parameter creation example. Terraform plans the creation of aws_ssm_parameter.example with type String and a sensitive value. The plan shows creation of arn, datatype, id, insecurevalue, keyid, name, tagsall, tier, type, value, version. After apply, the output ssm_param_value is set to set by terraform. Confirmation via the AWS CLI demonstrates the parameter exists:

aws ssm get-parameter --name example

The returned Parameter object shows Name example, Type String, Value set by terraform, Version 1, LastModifiedDate 2024-09-17T08:03:23.914000+10:00, ARN arn:aws:ssm:us-east-1:012345678910:parameter/example, DataType text. Simulating an external update changes the value in SSM. A subsequent terraform apply with a standard configuration reverts the value back to set by terraform and reports modifications complete. The value has reverted to set by terraform.

The benefit of a shared SSM Parameter is referencing the current value. When Terraform refreshes state, it pulls the current value of aws_ssm_parameter.example.value from SSM. This enables patterns where GitHub Actions build docker images, push to ECR, and update a current stable tag in SSM. Terraform can then pull this value when deploying an ECS task or Lambda function. Using an aws_ssm_parameter data source is possible but fails if the data source does not exist, preventing creation of dependent resources like the ECR repo until the parameter exists. That reintroduces manual creation. Regular rotation of credentials and secrets is security hygiene. Tracking infrastructure as code ensures consistency. Using lifecycle arguments for SSM Params allows both.

Managing SSM Parameters with Terraform and External Updates

The external update problem is common in real environments. Parameters that hold image tags, feature flags, or endpoint URLs are often updated by automation. If Terraform enforces a static value, deployments become brittle.

Impact layer: Teams experience unexpected rollbacks where a manually corrected parameter is overwritten by the next Terraform run. This creates operational incidents, forces developers to avoid Terraform, and leads to configuration drift between state and reality. Allowing external updates while keeping the resource managed preserves auditability.

Contextual layer: The pattern connects to broader IaC principles. Infrastructure as code is accepted best practice in the cloud. Terraform is the most popular cross cloud framework for infrastructure as code, but it presents challenges when dealing with resources updated by external processes. SSM Parameters are a prime example. The solution of lifecycle meta arguments is reusable for other resources with mutable fields.

The reference implementation demonstrates creation, confirmation, external mutation, and reversion. The full cycle illustrates why ignore changes is required.

Lifecycle Meta Arguments and Ignoring Value Changes

Terraform’s lifecycle meta argument controls create, update, and delete behavior. For SSM Parameters, the critical use is ignore_changes = [value]. This instructs Terraform to ignore differences between the state and the actual parameter value.

A module wrapper exposes this as ignore_value_changes = true. The module usage shows:

module "list" { source = "terraform-aws-modules/ssm-parameter/aws" ignore_value_changes = true name = "my-parameter-ignore-value-changes" value = "some-value" }

The module allows managing multiple resources with less code. One of multiple SSM Parameters can be created. Value type guesser is provided. Allow SSM Parameter to ignore changes in the value.

Impact layer: Engineers can define an initial value for bootstrapping and then allow operational teams to rotate secrets without Terraform conflicts. The resource remains in state, so name, type, and tags remain enforced.

Contextual layer: This pattern aligns with the SSM Parameter lifecycle and the broader need for shared mutable state. The module supports secure types, tiering, and pattern validation.

Terraform Module for SSM Parameters

The terraform-aws-modules/terraform-aws-ssm-parameter module creates AWS SSM Parameters on AWS. It supports string, secure string, and list types.

Example usage for a simple string:

module "string" { source = "terraform-aws-modules/ssm-parameter/aws" name = "my-parameter" value = "some-value" }

Secure type usage:

module "secret" { source = "terraform-aws-modules/ssm-parameter/aws" name = "my-secret-token" value = "secret123123!!!" secure_type = true }

List parameter:

module "list" { source = "terraform-aws-modules/ssm-parameter/aws" name = "my-list-parameter" values = ["item1", "item2"] }

The module also supports locals for batch creation:

locals { parameters = { "string_simple" = { value = "string_value123" } "string" = { type = "String" value = "string_value123" tier = "Intelligent-Tiering" allowed_pattern = "[a-z0-9_]+" } "secure" = { type = "SecureString" value = "secret123123!!!" tier = "Advanced" description = "My awesome password!" } "secure_encrypted_true" = { secure_type = true value = "secret123123!!!" key_id = "c938de44-1c09-4c91-89fd-b5881f06f317" } "list_as_autoguess_type" = { values = ["item1", "item2"] } } }

The module provides value type guesser, allowing type inference. It supports tier selection such as Intelligent-Tiering and Advanced. It supports allowedpattern for validation. It supports keyid for customer managed encryption.

Impact layer: Centralizing parameter creation reduces duplication. Teams avoid writing repetitive aws_ssm_parameter resources. The wrapper module reduces code volume and enforces consistent naming, tagging, and security settings.

Contextual layer: The module integrates with Terraform’s dependency graph. Parameters can be outputs for other modules that consume them for ECS task definitions, Lambda environment variables, or IAM policy conditions.

Centralized EC2 Patching with SSM Across Accounts

AWS Systems Manager simplifies patching automation and allows for the sharing of patching baselines from a central account to multiple child accounts. The blog walkthrough shows how to implement cross-account SSM patching using Terraform, enabling centralized control while allowing distributed execution across accounts.

Why this matters:

  • Centralized Control: Manage and enforce patch compliance policies from a single security or DevOps account.
  • Scalability: Easily extend the patching solution across dozens or hundreds of AWS accounts without manual setup.
  • Security and Compliance: Ensure consistent patching of vulnerabilities to meet regulatory standards e.g., CIS, HIPAA.
  • Cost Optimization: Avoid redundant configuration and tool sprawl using AWS-native features like SSM.
  • Audit Readiness: Maintain visibility and traceability of patching operations across your organization.

The architecture uses a master account that references an AWS default patch baseline. Child accounts use the AWS default patch baseline, tag EC2 instances, and automate patching with AWS Systems Manager.

Impact layer: Organizations reduce manual patching effort, improve compliance posture, and gain centralized reporting. Patch windows can be coordinated from a single location.

Contextual layer: Patching integrates with IAM roles, EC2 tagging, and SSM associations. Terraform codifies these relationships, making them repeatable.

Prerequisites and Terraform Checklist for Cross Account Patching

Before getting started, ensure you have the following:

  • Terraform >=1.0.0 installed on your local machine
  • AWS CLI configured for both central master and child accounts
  • IAM permissions to create and manage SSM and Amazon EC2 resources
  • Tags applied to Amazon EC2 instances to group them for patching e.g., Patch Group
  • A proper role in child accounts allowing SSM to run patching commands
  • Cross-account trust relationship configured if assuming roles between accounts

Terraform Checklist before writing infrastructure code:

  • Understand the Use Case
  • Define the objective e.g., centralized patching using AWS-managed patch baselines
  • Identify participating AWS accounts and their roles master and child
  • Identify Required Services
  • AWS Systems Manager SSM
  • AWS Identity and Access Management IAM
  • Amazon EC2 for tagging and patching targets
  • Design the Architecture
  • Master account uses AWS default patch baseline
  • Child accounts associate Amazon EC2 tags and automate patching
  • Define AWS IAM Requirements
  • AWS Systems Manager execution role for patching in child accounts
  • Role assumption policies between accounts
  • Write and Organize Terraform Code
  • Separate files modules for patch group and automation
  • Use variables.tf, main.tf, and outputs.tf for modular structure
  • Include tagging, versioning, and logging where possible
  • Testing & Validation
  • Ensure Amazon EC2 instances are tagged correctly
  • Simulate patch execution via manual association run or maintenance window

Impact layer: Skipping prerequisites leads to failed associations, permission errors, and incomplete patch coverage. The checklist forces explicit design decisions.

Contextual layer: The checklist maps directly to Terraform best practices for multi-account environments. It mirrors patterns used for centralized logging, security baselines, and cost allocation.

Architecture and IAM Design for Patching

The architecture diagram described in the reference shows Master Account referencing AWS Default Patch Baseline, and Child Accounts using AWS Default Patch Baseline, tagging EC2 Instances, and automating patching with AWS Systems.

Required IAM components include an SSM execution role for patching in child accounts and role assumption policies between accounts. The master account does not execute patches directly; it provides the baseline reference. Child accounts associate EC2 tags with the baseline and schedule automation.

The Terraform code for this scenario typically creates aws_ssm_association, aws_ssm_document, and CloudWatch log groups for session logging. The reference material for SSM session management shows resources:

  • awscloudwatchloggroup.ssmlog_group resource
  • awsssmassociation.main resource
  • awsssmdocument.custom resource
  • awsssmdocument.session_preferences resource

Variables include associationname, cloudwatchencryptionenabled, cloudwatchstreamingenabled, content, createcustomdocument, createsessionpreferences, documentformat.

The session document usage example:

aws ssm start-session \ --target "<instance_id_here>" \ --document-name "<name_of_created_session_document>"

Name and version requirements:

  • terraform >= 0.14.11
  • aws >= 5.15.0

Actual versions listed: aws 5.33.0. No modules.

Impact layer: Centralized patching reduces configuration sprawl. IAM roles ensure least privilege. Logging provides audit trails for compliance.

Contextual layer: The patching pattern complements SSM Parameter management. Both rely on Terraform to codify SSM resources, IAM roles, and associations, while allowing runtime mutability where needed.

Integration Patterns and Operational Considerations

Combining SSM Parameter management with patch automation creates a cohesive Systems Manager footprint managed by Terraform. Parameters can store the name of the patch baseline, the maintenance window schedule, or the tag key used for grouping instances. Terraform creates the resources, while external automation updates mutable values.

The lifecycle ignorechanges pattern prevents Terraform from reverting a parameter that is updated by a CI pipeline. The module wrapper reduces boilerplate for creating many parameters across environments. Cross-account patching uses Terraform to define associations and IAM roles once, then replicate across child accounts via Terraform workspaces or foreach.

Testing and validation steps include verifying EC2 instances are tagged correctly and simulating patch execution via manual association run or maintenance window. This ensures the Terraform applied configuration matches operational reality.

The combination of centralized control, scalability, and audit readiness makes the Terraform SSM approach suitable for regulated environments.

Conclusion

AWS Systems Manager Parameters and patching automation present distinct challenges for infrastructure as code. Parameters require protection from reversion when updated externally, achieved through lifecycle meta arguments and module options that ignore value changes. The terraform-aws-modules/terraform-aws-ssm-parameter module provides a wrapper for creating string, secure string, and list parameters with type guessing, tiering, and pattern validation. Centralized EC2 patching across accounts leverages SSM patch baselines, IAM role assumption, and EC2 tagging, all codified in Terraform with explicit prerequisites and a structured checklist. Together these patterns enable organizations to maintain infrastructure as code discipline while allowing operational mutability where necessary, supporting compliance, auditability, and scale without manual configuration sprawl.

Sources

  1. Proactive Ops
  2. Cloud That
  3. terraform-aws-modules terraform-aws-ssm-parameter
  4. Terraform Foundation terraform-aws-ssm

Related Posts