Synergizing Infrastructure Provisioning and Configuration Management with Terraform and Ansible

The modern landscape of Infrastructure as Code (IaC) and configuration management is dominated by two powerhouse tools: Terraform and Ansible. While they are often viewed as competitors because they occasionally overlap in functionality, the most sophisticated engineering workflows treat them as complementary components of a larger delivery pipeline. Terraform, developed by HashiCorp, is a declarative tool designed to manage the lifecycle of infrastructure across a vast array of cloud platforms, Kubernetes, and messaging systems like RabbitMQ. It utilizes the HashiCorp Configuration Language (HCL) to define the desired end-state of a system. In contrast, Ansible excels at the imperative task of configuring software and managing the internal state of an operating system once the hardware or virtual machine exists.

When these tools are used in isolation, organizations often face a "gap" between the moment a virtual machine is powered on and the moment it is ready to serve production traffic. This gap is where the synergy between Terraform and Ansible becomes critical. Using Terraform to handle the foundational layer—networking, storage, and compute—and Ansible to handle the application layer—software installation, security hardening, and service orchestration—creates an end-to-end elevated workflow. This approach avoids the fragility of Terraform's internal provisioners, which are widely regarded as a last resort by HashiCorp, and the complexity of using Ansible for heavy-lift infrastructure provisioning, which often requires an excessive amount of custom code for highly specific cloud configurations.

The Architecture of Integrated Workflows

The most effective pattern for integrating these two technologies is a staged pipeline where Terraform acts as the foundation and Ansible acts as the decorator.

The Provisioning Phase with Terraform

Terraform operates as an orchestration engine that focuses on the "outer" shell of the environment. Its primary goal is to ensure that the infrastructure matches the state defined in HCL files. Because Terraform maintains a state file—a JSON representation of the managed resources—it can track exactly what has been deployed. In a typical integrated workflow, Terraform is used to deploy:

• Cloud networking (VPCs, Subnets, Gateways)
• Compute instances (AWS EC2, vSphere VMs, Azure VMs)
• Storage volumes and managed databases
• Security groups and firewall rules
• Identity and Access Management (IAM) roles

By using a declarative approach, engineers can define modules for reusability and employ loops and conditionals to scale the environment. For instance, a single HCL block can deploy multiple instances of a virtual machine across different availability zones, ensuring high availability before Ansible ever connects to the machines.

The Configuration Phase with Ansible

Once Terraform has successfully instantiated the hardware, Ansible takes over to perform "Day 1" and "Day 2" operations. Ansible's agentless nature makes it ideal for this, as it only requires SSH access and Python on the target host. While Terraform knows that a server exists, Ansible knows how to make that server a functional web server or database node.

The transition from Terraform to Ansible generally happens through one of three methods: manual invocation, dynamic inventory scripts, or sophisticated orchestration platforms like Spacelift. The goal is to move from the "infrastructure" state to the "application" state without manual intervention.

Technical Implementation: A Detailed AWS Example

To understand how these tools interact in a real-world scenario, consider a deployment of three Ubuntu EC2 instances in the eu-west-1 region.

Terraform Configuration Logic

The Terraform side of the operation requires a provider block for AWS and a data source to fetch the latest Ubuntu AMI. The following configuration demonstrates the creation of these resources:

```hcl
provider "aws" {
region = "eu-west-1"
}

data "awsami" "ubuntu" {
mostrecent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
owners = ["099720109477"] #canonical
}

locals {
instances = {
instance1 = {
ami = data.awsami.ubuntu.id
instancetype = "t2.micro"
}
instance2 = {
ami = data.awsami.ubuntu.id
instancetype = "t2.micro"
}
instance3 = {
ami = data.awsami.ubuntu.id
instancetype = "t2.micro"
}
}
}

resource "awskeypair" "sshkey" {
keyname = "ec2"
publickey = file(var.publickey)
}

resource "awsinstance" "this" {
foreach = local.instances
ami = each.value.ami
instancetype = each.value.instancetype
keyname = awskeypair.sshkey.keyname
associatepublicipaddress = true
tags = {
Name = each.key
}
}
```

In this implementation, the aws_instance resource uses a for_each loop over the local.instances map to create three distinct servers. A critical step here is the associate_public_ip_address = true setting. By assigning public IPs, the workflow accelerates the ability of Ansible to connect to the instances immediately after they boot, bypassing the need for complex bastion host configurations during the initial setup phase.

Ansible Configuration Logic

After the terraform apply is complete, Ansible is invoked to transform these raw Ubuntu instances into functional web servers. The typical Ansible playbook in this scenario would execute the following steps:

• Installation of the Apache Web Server
• Configuration of the system firewall to open port 80
• Installation of Docker (providing a foundation for future containerized workloads)
• Configuration of DNS settings via the resolv.conf file
• Deployment of a custom index.html page

To ensure the index page is unique to each server, Ansible utilizes Jinja2 templates. The apache-web-server role contains an index.html template that dynamically inserts the machine's hostname before copying the file to the target host, allowing engineers to verify which specific instance they are accessing.

Advanced Integration with VMware vSphere

The synergy between these tools is also highly effective in on-premises environments using VMware vSphere. In this scenario, the complexity increases as IP management and secret handling become more prominent.

Secret Management and Variable Handling

When deploying to vSphere, security is paramount. A common pattern involves splitting variables into two categories: general configuration and sensitive secrets.

• secret.tfvars: This HCL-formatted file contains the sensitive credentials required for the Terraform provider to authenticate with the vCenter server.
• variables.yml: This file is used by Ansible for software configuration. To protect this data, it is encrypted using the ansible-vault command.

The specific variables required for a vSphere deployment include:

• vsphere_user: The username for the vCenter account.
• vsphere_password: The password for the vCenter account.
• vsphere_server: The IP address or Fully Qualified Domain Name (FQDN) of the server.
• vsphere_vm_firmware: Defaults to bios.
• ssh-pub-key: The public key of a service account that allows Ansible to establish SSH connections.
• service_account_username: The OS-level username for the service account.
• service_account_password: The OS-level password for the service account.

To handle the decryption of variables.yml during the Ansible run, a .vault_pass.txt file may be used, containing a single line of text with the vault password.

Execution Workflow for vSphere

In a vSphere-based example, Terraform uses the vsphere_virtual_machine resource to create five virtual machines from a predefined template. It also injects the SSH public key of the service account into each host to ensure seamless Ansible access. While the example uses statically assigned IP addresses—which are hard-coded into the Terraform provisioner—this can be adapted to dynamic DHCP assignments.

The operational sequence to execute this workflow is as follows:

1. Initialize the environment:
   terraform init -var-file="secret.tfvars"

2. Create the execution plan:
   terraform plan -out da-compute-apache-web-server.tfplan -var-file="secret.tfvars"

3. Apply the changes:
   terraform apply -var-file="secret.tfvars"

Bridging the Gap: State Parsing and Dynamic Inventories

One of the primary challenges in using these tools together is telling Ansible where the new servers are. Because cloud IPs are often dynamic, a static inventory file is insufficient.

The State File Approach

When using a local backend, Terraform stores its state in a JSON file. This file contains the public and private IP addresses of all created resources. Some engineers use tools to parse this JSON file and convert it directly into an Ansible inventory. For example, a terraform-inventory tool can transform the state file into a format Ansible understands:

```text
[all]
52.51.215.84

[all:vars]

[server]
52.51.215.84

[server.0]
52.51.215.84

[typeawsinstance]
52.51.215.84

[name_c10k server]
52.51.215.84

[%_1]
52.51.215.84
```

Alternatively, custom Python scripts such as terraform.py can be used to generate a host file:

```text

begin hosts generated by terraform.py

52.51.215.84 C10K Server

end hosts generated by terraform.py

```

The Provisioner Dilemma

There have been attempts to create native Ansible provisioners within Terraform to make the transition seamless. For instance, projects like terraform-provisioner-ansible aim to allow a syntax like this:

hcl provisioner "ansible" { plays { playbook = "./provision.yml" hosts = ["${self.public_ip}"] } become = "yes" local = "yes" }

However, these third-party plugins often struggle with maintenance and compatibility with the current Terraform plugin system. The industry standard remains separating the two tools into distinct stages of a pipeline.

Emerging Technologies: Terraform Actions and AAP

To resolve the fragmentation caused by manual invocations, HashiCorp introduced "Terraform Actions." This represents a shift toward a unified infrastructure workflow.

Defining Terraform Actions

Terraform actions are pre-set operations built into providers that enable "Day 2" management operations. Unlike the standard plan and apply cycle, which focuses on the creation and destruction of resources (CRUD), actions can be invoked:

• Pre-CRUD events: Before a resource is created, read, updated, or destroyed.
• Post-CRUD events: After a resource has been modified.
• Ad hoc via CLI: Outside of the traditional state-management cycle.

Integration with Ansible Automation Platform (AAP)

The introduction of the AAP Terraform provider allows for a high-degree of automation. By using Terraform actions, a terraform apply can dispatch an event that activates the Event Driven Automation (EDA) capability of Ansible. This means that the moment Terraform completes the provisioning of a resource, it can trigger a complex, dynamic automation workflow in Ansible without any manual trigger or external scripting. This eliminates the need for fragmented workflows where teams must manually invoke Lambda functions or send SNS notifications to alert configuration tools that the infrastructure is ready.

Comparative Analysis of Approaches

The following table summarizes the different methodologies for combining Terraform and Ansible.

Conclusion: The Path to Absolute Automation

The integration of Terraform and Ansible is not merely about running one after the other; it is about establishing a clear boundary between the lifecycle of the hardware and the lifecycle of the software. Terraform provides the stability and predictability of a declarative state, ensuring that the underlying network and compute resources are consistent. Ansible provides the flexibility and granularity of an imperative configuration tool, ensuring that the software is tuned and secured.

The shift toward "Terraform Actions" and Event-Driven Automation (EDA) marks the evolution of this relationship. By moving away from the "provisioner" model—which HashiCorp explicitly advises against—and moving toward an event-based trigger system, organizations can achieve a true "single source of truth." In this mature model, a single change to an HCL file can ripple through the entire stack: updating a VM's size in Terraform, triggering an event in AAP, and resulting in a re-configured application environment via Ansible, all without manual intervention. This synergy reduces operational overhead, eliminates the risk of manual configuration drift, and provides a scalable blueprint for any modern cloud-native architecture.

Sources

1. Using Terraform and Ansible Together - Spacelift
2. Unifying Infrastructure Provisioning and Configuration Management - HashiCorp
3. Terraform plus Ansible Example - Cisco DevNet
4. Terraform and Ansible Integration - Alex Dzyoba