Terraform AWS SSM Parameter Management With External Update Tolerance And Post-EC2 Association Timing

Infrastructure as code adoption makes Terraform the most popular cross cloud framework for infrastructure management. AWS Systems Manager Parameter Store provides a central location for configuration data and secrets. The intersection of Terraform and AWS SSM introduces specific behavioral patterns around state synchronization, external modifications, and timing dependencies during instance provisioning. The reference material demonstrates how Terraform lifecycle meta arguments can preserve externally updated parameter values, how the terraform-aws-modules/ssm-parameter module abstracts parameter creation, and how timing controls like time_sleep resolve SSM association failures on newly created EC2 instances.

The core challenge described is that SSM Parameters can be modified by other applications or workflows outside Terraform control. When Terraform is used to set an initial value, each terraform apply execution will revert the value to the initial value unless explicit lifecycle controls are applied. The article Managing AWS SSM Parameters with Terraform with External Updates explores how to use Terraform to create and manage SSM Parameters while allowing external updates seamlessly. The goal is to ensure infrastructure is tracked while Terraform state stays in sync with reality. The demonstration uses Terraform's lifecycle meta argument to solve the drift problem.

The operational flow begins with resource creation. Terraform will perform the following actions for a new parameter:

```

awsssmparameter.example will be created

  • resource "awsssmparameter" "example" {
  • arn = (known after apply)
  • data_type = (known after apply)
  • id = (known after apply)
  • insecure_value = (known after apply)
  • key_id = (known after apply)
  • name = "example"
  • tags_all = (known after apply)
  • tier = (known after apply)
  • type = "String"
  • value = (sensitive value)
  • version = (known after apply)
    }
    Plan: 1 to add, 0 to change, 0 to destroy.
    Changes to Outputs:
  • ssmparamvalue = "set by terraform"
    ```

The apply output confirms creation:

aws_ssm_parameter.example: Creating... aws_ssm_parameter.example: Creation complete after 2s [id=example] Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Outputs: ssm_param_value = "set by terraform"

Verification with AWS CLI shows the stored value:

aws ssm get-parameter --name example

The returned Parameter block contains:

{ "Parameter": { "Name": "example", "Type": "String", "Value": "set by terraform", "Version": 1, "LastModifiedDate": "2024-09-17T08:03:23.914000+10:00", "ARN": "arn:aws:ssm:us-east-1:012345678910:parameter/example", "DataType": "text" } }

The value has been set to set by terraform. The real-world consequence for operators is that the initial seed value is now persisted in Parameter Store and can be referenced by other services. When GitHub Actions builds docker images and pushes them to ECR, Actions could update the current stable tag in SSM. Terraform could pull this value when deploying ECS task or Lambda function. This pattern decouples deployment pipelines from infrastructure pipelines while maintaining a single source of truth.

Terraform Lifecycle Meta Argument for SSM Parameters

The lifecycle meta argument is the primary mechanism for preventing Terraform from overwriting externally managed values. Without lifecycle controls, an external update to the parameter is treated as drift and Terraform will revert the change on the next apply.

The simulation of external change uses the AWS CLI to update the value:

aws ssm get-parameter --name example

To simulate our value being updated by another process, we can update the value using the AWS CLI. The external process modifies the parameter value from set by terraform to set by cli.

When Terraform refreshes state, it detects the change:

```
awsssmparameter.example: Refreshing state... [id=example]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:

awsssmparameter.example has changed

~ resource "awsssmparameter" "example" {
id = "example"
name = "example"
~ value = (sensitive value)

(9 unchanged attributes hidden)

}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.
```

The plan shows:

Changes to Outputs: ~ ssm_param_value = "set by terraform" -> "set by cli" You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

The apply execution then confirms retention:

terraform apply -auto-approve

aws_ssm_parameter.example: Refreshing state... [id=example] Note: Objects have changed outside of Terraform

Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: ssm_param_value = "set by cli"

The value remains unchanged. If we remove the lifecycle argument, we will see the value reverts. The impact layer for practitioners is that credentials and secrets can be rotated outside Terraform without causing configuration drift failures. Regular rotation of credentials and other secrets is an important security hygiene habit. Tracking and managing infrastructure as code ensures consistency in environments. Using lifecycle arguments for SSM Params allows us to do both, without conflict or surprises.

The benefit of using a shared SSM Parameter like this is that we can reference the current value of the param. When terraform refreshes its state, it pulls the current value of awsssmparameter.example.value from SSM. This creates a read-through pattern where Terraform becomes aware of external changes without attempting to overwrite them.

A common alternative is the awsssmparameter data source. The reference material notes a downside: Terraform fails if the data source doesn’t exist. This means we can’t create the ECR repo and other resources until the param exists. We’ve gone full circle and are back to creating the value manually, which we want to avoid.

Managing SSM Parameters Creation With Terraform Modules

The terraform-aws-modules/ssm-parameter/aws module provides a wrapper for creating AWS SSM Parameters with less code. The module supports creating one of multiple SSM Parameters, value type guesser, and allowing SSM Parameter to ignore changes in the value.

Basic usage examples from the module documentation:

module "string" { source = "terraform-aws-modules/ssm-parameter/aws" name = "my-parameter" value = "some-value" }

module "secret" { source = "terraform-aws-modules/ssm-parameter/aws" name = "my-secret-token" value = "secret123123!!!" secure_type = true }

module "list" { source = "terraform-aws-modules/ssm-parameter/aws" name = "my-list-parameter" values = ["item1", "item2"] # "values" not "value" }

module "list" { source = "terraform-aws-modules/ssm-parameter/aws" ignore_value_changes = true name = "my-parameter-ignore-value-changes" value = "some-value" }

The ignorevaluechanges flag directly implements the lifecycle ignore_changes pattern for value attributes. The module abstracts the need to write explicit lifecycle blocks for each parameter.

The locals example demonstrates complex parameter definitions:

```
locals {
parameters = {

#

String

#

"stringsimple" = {
value = "string
value123"
}
"string" = {
type = "String"
value = "stringvalue123"
tier = "Intelligent-Tiering"
allowed
pattern = "[a-z0-9_]+"
}

#

SecureString

#

"secure" = {
type = "SecureString"
value = "secret123123!!!"
tier = "Advanced"
description = "My awesome password!"
}
"secureencryptedtrue" = {
securetype = true
value = "secret123123!!!"
key
id = "c938de44-1c09-4c91-89fd-b5881f06f317"
}

#

StringList

#

"listasautoguesstype" = {
values = ["item1", "item2"]
}
"list
asjsonencodedstring" = {
type = "StringList"
value =
```

The module supports tier selection such as Intelligent-Tiering and Advanced. Allowed pattern validation is supported for String parameters. SecureString parameters can be tied to a specific KMS keyid for encryption. The contextual layer connects this to compliance requirements where different tiers offer different throughput and storage limits, and keyid binding enforces customer managed key policies.

Parameter Type Configurations and Attribute Patterns

SSM Parameter Store supports multiple types. The reference material shows String, SecureString, and StringList configurations.

String parameters can be created with simple value assignment or with explicit type, tier, and allowedpattern constraints. The stringsimple definition shows minimal configuration:

value = "string_value123"

The expanded string definition adds:

type = "String" value = "string_value123" tier = "Intelligent-Tiering" allowed_pattern = "[a-z0-9_]+"

SecureString parameters provide encryption at rest. The secure definition includes:

type = "SecureString" value = "secret123123!!!" tier = "Advanced" description = "My awesome password!"

The secureencryptedtrue variant uses securetype flag and keyid:

secure_type = true value = "secret123123!!!" key_id = "c938de44-1c09-4c91-89fd-b5881f06f317"

StringList parameters support multiple values. The listasautoguess_type uses values array:

values = ["item1", "item2"]

The listasjsonencoded_string shows type StringList with value assignment.

The impact for operators is that value type guessing reduces boilerplate for simple string parameters, while explicit type declarations enforce validation and prevent accidental misclassification of secrets.

External Updates and State Synchronization Behavior

The Terraform workflow for SSM parameters with external updates involves three phases: creation, external modification, and state refresh.

Creation phase establishes the resource and outputs the initial value:

Outputs: ssm_param_value = "set by terraform"

External modification phase simulates a real world process updating the parameter:

aws ssm get-parameter --name example

The parameter value changes outside Terraform control.

Refresh phase detects drift:

aws_ssm_parameter.example: Refreshing state... [id=example] Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan.

The plan proposes to update outputs without modifying infrastructure:

Changes to Outputs: ~ ssm_param_value = "set by terraform" -> "set by cli"

Applying the plan saves the new output value to state:

Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: ssm_param_value = "set by cli"

The contextual connection is that this pattern enables GitOps workflows where CI systems update SSM parameters and Terraform consumes the latest value during deployment without reverting changes.

The module's ignorevaluechanges flag provides declarative control:

ignore_value_changes = true name = "my-parameter-ignore-value-changes" value = "some-value"

This prevents Terraform from attempting to write back the initial value after external changes.

Timing Dependencies For SSM Associations On New EC2 Instances

SSM associations on newly created EC2 instances can fail due to timing issues. The instance must pass systems checks and be ready for SSM communication before associations are applied.

A solution uses the terraform time_sleep resource to introduce a delay:

resource "time_sleep" "wait_60_seconds" { depends_on = [aws_instance.ec2-instance] create_duration = "60s" }

You can then depend on the timesleep resource this will then create the awsssm_association resource after 1 minute.

The SSM Run command example configures CloudWatch Agent:

resource "aws_ssm_association" "cloudwatch-config" { name = "AmazonCloudWatch-ManageAgent" targets { key = "InstanceIds" values = [aws_instance.ec2-instance.id] # Use the correct instance ID from aws_instance } parameters = { action = "configure" mode = "ec2" optionalConfigurationSource = "ssm" optionalConfigurationLocation = "CWA_config" optionalRestart = "yes" } depends_on = [ aws_ssm_association.cloudwatch-agent, time_sleep.wait_60_seconds ] }

The depends_on meta argument ensures the association is created only after the instance exists and the sleep duration elapses. The real-world consequence is successful installation and configuration of the CloudWatch agent using Terraform and AWS Systems Manager without race conditions.

It appears that you're encountering a timing issue when trying to install and configure the CloudWatch agent using Terraform and AWS Systems Manager.

The timesleep resource creates a deliberate pause of around a minute which gave the instance a chance to pass all systems checks then all the awsssm_association resources created successfully.

Module Requirements And Version Constraints

The Terraform Foundation terraform-aws-ssm module defines explicit provider version constraints.

Name Version table:

Name Version
terraform >= 0.14.11
aws >= 5.15.0

The resolved version example shows:

Name Version
aws 5.33.0

No modules are declared.

Resources created by the module include:

Name Type
awscloudwatchloggroup.ssmlog_group resource
awsssmassociation.main resource
awsssmdocument.custom resource
awsssmdocument.session_preferences resource

These resources support session logging and custom document creation.

Variable definitions include:

Name Description Type Default Required
association_name The descriptive name for the association. string null no
cloudwatchencryptionenabled If set to true, the log group you specified in the cloudWatchLogGroupName input must be encrypted. bool true no
cloudwatchstreamingenabled If set to true, a continual stream of session data logs are sent to the log group you specified in the cloudWatchLogGroupName input. If set to false, session logs are sent to the log group you specified in the cloudWatchLogGroupName input at the end of your sessions. bool true no
content The JSON or YAML content of the document. string "" no
createcustomdocument Specify whether to create ssm document with custom settings bool false no
createsessionpreferences Whether to create session preferences string false no
document_format The format of the document

The version constraints ensure compatibility with Terraform 0.14.11 and later and AWS provider 5.15.0 and later. The impact is that teams using older Terraform versions must upgrade to avoid provider incompatibilities.

SSM Session Management And Document Configuration

Session management via SSM requires starting a session with target instance and document name:

aws ssm start-session \ --target "<instance_id_here>" \ --document-name "<name_of_created_session_document>"

The module creates CloudWatch log groups for session logs and SSM documents for custom settings. The cloudwatchencryptionenabled flag controls whether logs must be encrypted. The cloudwatchstreamingenabled flag controls continual streaming versus end-of-session delivery.

The associationname variable allows descriptive naming for SSM associations. The createcustom_document flag controls whether a custom SSM document is created. The content variable accepts JSON or YAML document content.

The contextual layer connects session logging to audit requirements where continuous streaming provides real-time monitoring while end-of-session delivery reduces API calls.

Conclusion

The integration of Terraform with AWS SSM Parameter Store creates a control plane where initial provisioning and ongoing drift tolerance coexist. The lifecycle meta argument and ignorevaluechanges module parameter provide the technical mechanism to allow external processes to update SSM parameters without Terraform reverting those changes. The demonstrated workflow of creating a parameter, externally updating it via AWS CLI, and then refreshing Terraform state shows that outputs can be updated to reflect reality without destructive writes.

The terraform-aws-modules/ssm-parameter module abstracts repetitive parameter definitions, supports value type guessing, and offers explicit ignorevaluechanges control. Parameter configurations span String with Intelligent-Tiering and allowed patterns, SecureString with Advanced tier and KMS key binding, and StringList with multi-value support. These options map directly to operational needs for performance, validation, encryption, and list management.

Timing dependencies for SSM associations on EC2 instances highlight the need for explicit ordering. The timesleep resource provides a pragmatic delay to allow instance systems checks to complete before awsssm_association resources are applied. This resolves race conditions where SSM commands would otherwise fail on newly created instances.

Version constraints for the Terraform Foundation SSM module ensure compatibility with Terraform >= 0.14.11 and AWS provider >= 5.15.0. Resources such as awscloudwatchloggroup.ssmloggroup, awsssmassociation.main, awsssmdocument.custom, and awsssmdocument.sessionpreferences enable session logging and custom document workflows. Variables for association naming, CloudWatch encryption and streaming, document content, and creation flags provide fine-grained control over session behavior.

Together these patterns form a coherent approach to managing AWS SSM with Terraform while preserving external update workflows and ensuring reliable association timing during instance provisioning.

Sources

  1. Managing AWS SSM Parameters with Terraform with External Updates
  2. Terraform module which creates AWS SSM Parameters on AWS
  3. Terraform Foundation terraform-aws-ssm
  4. Running SSM commands after EC2 instance is created Terraform

Related Posts