The debate between infrastructure-as-code tools often presents a false dichotomy, forcing practitioners to choose exclusively between Terraform’s provider-agnostic flexibility or AWS CloudFormation’s native deep integration with the broader AWS ecosystem. However, modern cloud engineering demands a pragmatic approach where these tools coexist rather than compete. Terraform, while not natively designed as a wrapper for AWS-specific orchestration primitives, offers first-class support for managing CloudFormation stacks and StackSets through specific resources. This hybrid methodology allows engineering teams to leverage the best of both worlds: the robust state management, planning capabilities, and multi-provider support of Terraform, combined with the centralized multi-account deployment capabilities of AWS CloudFormation StackSets. Understanding how to bridge these two paradigms is essential for organizations maintaining legacy CloudFormation templates, deploying consistent configurations across AWS Organizations, or migrating gradually from pure CloudFormation environments to a Terraform-centric infrastructure workflow.
The Architectural Rationale for Hybrid Deployment
Managing one infrastructure-as-code tool with another might initially appear counterintuitive, suggesting a layer of unnecessary complexity. Yet, practical constraints often dictate this hybrid approach. Organizations frequently possess existing CloudFormation templates that have matured over years of production use, undergoing rigorous testing and validation in specific business contexts. Rewriting these templates into native Terraform resources can introduce significant risk, potentially breaking subtle behaviors or configurations that are unique to CloudFormation’s dependency graph and intrinsic functions. Furthermore, certain AWS features may only be available through CloudFormation at the time of implementation, or specific AWS-native capabilities like StackSets provide a level of orchestration that Terraform does not natively replicate in its core design.
Terraform was architected to manage resources directly on a per-configuration basis, focusing on a flat resource graph rather than the hierarchical, account-wide orchestration that defines CloudFormation StackSets. In CloudFormation, StackSets are a core feature that allows for the centralized deployment and management of identical stack templates across multiple AWS accounts and regions. Terraform bridges this gap not by replicating the internal logic of StackSets, but by exposing the aws_cloudformation_stack_set resource. This resource acts as a wrapper around AWS’s existing StackSets service. When a team defines this resource in Terraform, they are essentially instructing Terraform to interact with the AWS API to create, update, or delete a StackSet. The actual orchestration of deployments across accounts remains within AWS CloudFormation’s domain, while Terraform serves as the single control plane for the lifecycle of that orchestration object. This setup allows teams to maintain a consistent workflow in Terraform for multi-service and multi-cloud environments while leveraging AWS-native capabilities where they are indispensable.
Defining and Deploying CloudFormation Stacks via Terraform
The primary mechanism for deploying individual CloudFormation stacks using Terraform is the aws_cloudformation_stack resource. This resource allows users to define a stack’s lifecycle entirely within Terraform configuration files. The critical aspect of this resource is the template_body parameter, which accepts the CloudFormation template. This template can be provided in two primary formats: as an inline string within the Terraform configuration or as an external file reference.
For complex templates, storing the YAML or JSON inline within the HCL code becomes unwieldy and difficult to maintain. Best practices dictate storing these templates in version control alongside the Terraform code. This separation of concerns ensures that the infrastructure logic remains readable and that the CloudFormation templates can be validated independently using standard CloudFormation linters or preview tools. When referencing an external file, the file() function is used, as demonstrated in the following pattern:
```hcl
resource "awscloudformationstack" "vpcstack" {
name = "terraform-managed-cfn-vpc"
templatebody = file("${path.module}/templates/vpc.yaml")
parameters = {
VpcCidr = "10.0.0.0/16"
}
tags = {
ManagedBy = "Terraform"
}
}
```
In this configuration, the vpc.yaml file contains the standard CloudFormation structure. A typical template for a VPC might include the AWSTemplateFormatVersion, resources for the AWS::EC2::VPC, and outputs such as the VPC ID. The CloudFormation template structure remains standard AWS YAML or JSON, independent of Terraform syntax. For example, a template defining a VPC and its outputs would look like this:
yaml
AWSTemplateFormatVersion: '2010-09-09'
Description: 'VPC created via Terraform'
Resources:
MyVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VpcCidr
Outputs:
VpcId:
Description: "The VPC ID"
Value: !Ref MyVPC
Terraform then parses this, submits it to AWS CloudFormation, and tracks the resulting stack ID in its state file. This allows subsequent Terraform resources to depend on the existence of the CloudFormation stack and to reference its outputs directly.
Bridging Systems: Referencing CloudFormation Outputs
One of the most powerful patterns in hybrid infrastructure is the ability to create resources in CloudFormation and immediately consume their outputs within the rest of the Terraform configuration. This eliminates the need for hardcoded values or manual lookups. CloudFormation stack outputs are the bridge between the isolated CloudFormation ecosystem and the broader Terraform state.
When a CloudFormation stack completes its creation phase, any defined outputs become available to the aws_cloudformation_stack resource in Terraform. These outputs can be accessed using the syntax aws_cloudformation_stack.<name>.outputs["<OutputKey>"]. This capability is particularly useful for integrating infrastructure components that are only available via CloudFormation with services managed natively by Terraform.
Consider a scenario where a database cluster is deployed via a CloudFormation template due to complex dependency requirements or specific parameter handling that is easier to manage in YAML. The database endpoint is exposed as an output. Terraform can then create an SSM parameter or an ECS task definition that consumes this value dynamically.
```hcl
Create infrastructure via CloudFormation
resource "awscloudformationstack" "databasestack" {
name = "database-stack"
templatebody = file("${path.module}/templates/database.yaml")
parameters = {
DBInstanceClass = "db.r5.large"
DBName = "myapp"
}
capabilities = ["CAPABILITY_IAM"]
}
Use CloudFormation outputs in other Terraform resources
resource "awsssmparameter" "dbendpoint" {
name = "/myapp/database/endpoint"
type = "String"
value = awscloudformationstack.databasestack.outputs["DBEndpoint"]
}
Reference in an application configuration
resource "awsecstaskdefinition" "app" {
family = "my-app"
containerdefinitions = jsonencode([
{
name = "app"
image = "my-app:latest"
environment = [
{
name = "DBHOST"
value = awscloudformationstack.databasestack.outputs["DBEndpoint"]
}
]
}
])
}
```
In this example, the DBEndpoint output from the CloudFormation stack is seamlessly integrated into the SSM parameter and the ECS task definition. This ensures that if the database endpoint changes due to a stack update, Terraform will detect the drift in the output value and update the dependent resources accordingly during the next plan and apply cycle. This tight coupling ensures consistency and reduces configuration drift.
Handling Complex Scenarios: IAM Capabilities and Timeouts
When working with CloudFormation stacks through Terraform, specific configuration options are critical to prevent deployment failures. CloudFormation has strict security measures regarding the creation of IAM resources. If a template includes resources of type AWS::IAM::*, the stack creation process requires explicit acknowledgment of this risk. In Terraform, this is handled via the capabilities argument.
If a template creates IAM roles, policies, or users, the capabilities list must include CAPABILITY_IAM. If the template uses dynamic references or requires access to other resources in different accounts, CAPABILITY_NAMED_IAM may also be required. Failing to specify these capabilities will result in a failure during the terraform apply step, as AWS will reject the stack creation request. It is a common source of error for developers who port templates without understanding these security safeguards.
Additionally, complex CloudFormation stacks, particularly those provisioning large numbers of instances or performing lengthy wait conditions, can take significant amounts of time to create. By default, Terraform has a timeout period for resource creation. If the stack creation exceeds this limit, Terraform will mark the operation as failed, even though the AWS CloudFormation process might still be running successfully. To mitigate this, the timeout_in_minutes argument should be explicitly set to a value appropriate for the complexity of the stack. This prevents unnecessary retries and state inconsistencies caused by premature timeout failures.
Scaling with StackSets for Multi-Account Organizations
For enterprises operating with AWS Organizations, deploying identical infrastructure configurations across dozens or hundreds of accounts is a common requirement. This is where CloudFormation StackSets become indispensable, and where Terraform’s aws_cloudformation_stack_set resource proves its value. StackSets allow for the centralized management of a single template across multiple target accounts and regions.
Unlike the single-account aws_cloudformation_stack, the StackSet resource requires the specification of an aws_organizations_account or aws_organizations_organizational_unit as targets. This enables the deployment of the stack to a specific set of accounts defined within the AWS Organization. The configuration typically involves defining the template body, the parameters, and the target accounts or organizational units.
```hcl
resource "awscloudformationstackset" "examplestackset" {
name = "example-stack-set"
templatebody = file("templates/example.yaml")
parameters = {
InstanceType = "t2.micro"
KeyName = "owntest"
}
# Target the specific organizational unit
stacksetinstanceregions = ["us-east-1"]
stacksetinstanceorganizationalunits = {
organizationalunit_ids = ["ou-abc123def456"]
}
}
```
In this configuration, Terraform manages the lifecycle of the StackSet. AWS CloudFormation handles the actual propagation of the stack to the accounts within the specified organizational unit in the us-east-1 region. This hybrid approach allows teams to use Terraform for their standard infrastructure management while relying on AWS’s native multi-account orchestration for scalable deployments. It avoids the need to write complex Terraform loops or provider aliases to manage multi-account credentials, which can be fragile and difficult to maintain.
Importing Legacy Stacks and Migration Strategies
Organizations transitioning from a pure CloudFormation environment to a Terraform-centric workflow often find themselves with existing stacks that they wish to manage via Terraform. This is achieved through the terraform import command. The process involves two main steps: defining the resource block in the Terraform configuration and then importing the existing stack’s ID into the Terraform state.
First, the developer must write a resource block that matches the existing stack’s configuration. This block does not need to be complete initially, but it must include the name and template_body (or template_body_json) that match the existing stack.
hcl
resource "aws_cloudformation_stack" "existing_stack" {
name = "my-existing-stack"
template_body = file("${path.module}/templates/existing.yaml")
}
Once the configuration is in place, the import command is executed using the stack name or ID:
bash
terraform import aws_cloudformation_stack.existing_stack my-existing-stack
After the import, the stack is managed by Terraform. Subsequent changes to the template body or parameters in the Terraform file will trigger updates to the existing CloudFormation stack. This migration path allows for a gradual transition. Teams can start by importing critical legacy stacks, then gradually rewrite new infrastructure in native Terraform resources, and eventually decommission the CloudFormation templates once they are no longer needed. This strategy minimizes risk and allows for parallel operation of both tools during the transition period.
Best Practices for Hybrid Workflows
To ensure stability and maintainability in a hybrid Terraform and CloudFormation environment, several best practices should be adhered to. First, always store CloudFormation templates in version control. Keeping the YAML or JSON templates alongside the Terraform code ensures that the infrastructure is fully reproducible and that changes to the template are tracked in the same commit history as changes to the Terraform configuration.
Second, always specify capabilities when IAM resources are involved. Omitting the capabilities argument is a frequent cause of deployment failures and should be avoided in production pipelines.
Third, use appropriate timeouts for complex stacks. Setting timeout_in_minutes to a value that reflects the expected creation time of the stack prevents false negatives in CI/CD pipelines.
Fourth, leverage StackSets for multi-account deployments. When working with AWS Organizations, attempting to manage multiple accounts via Terraform provider aliases can lead to complex state files. Using aws_cloudformation_stack_set delegates the multi-account complexity to AWS, simplifying the Terraform configuration.
Finally, plan migrations carefully. Import existing stacks before attempting to replace them with native Terraform resources. This ensures that the infrastructure is under Terraform’s state management before any changes are made, preventing accidental deletions or drift.
Conclusion
Managing CloudFormation stacks and StackSets through Terraform represents a sophisticated approach to infrastructure-as-code that acknowledges the strengths of both tools. By using the aws_cloudformation_stack and aws_cloudformation_stack_set resources, teams can retain the benefits of CloudFormation’s native orchestration and multi-account capabilities while gaining the superior state management, planning, and multi-provider flexibility of Terraform. This hybrid model is particularly beneficial for organizations with mature CloudFormation libraries, those requiring specific AWS-native features, or those undergoing gradual migrations. The key to success lies in proper configuration, including the correct use of IAM capabilities, timeouts, and version control for templates, as well as a clear strategy for importing and managing existing infrastructure. By bridging the gap between these two leading tools, engineering teams can build more resilient, scalable, and maintainable infrastructure in the AWS ecosystem.