Terraform VPC Endpoints for Air-Gapped AWS Environments

Published by TechMarmot Editorail Staff on | Category Terraform

Running Terraform against an AWS environment that has no path to the public internet requires private connectivity to AWS control plane and data plane services. VPC Endpoints provide that private connectivity using AWS PrivateLink and Elastic Network Interfaces, allowing Terraform to interact with AWS services without internet gateways or NAT. In an air-gapped setup the endpoints become the backbone for state storage, state locking, and provider operations. This guide covers the essential VPC endpoints required for Terraform and how to configure them with Terraform code and AWS CLI steps.

What Are VPC Endpoints

VPC Endpoints allow AWS resources in your VPC to privately connect to AWS services without going over the public internet. They use the AWS PrivateLink service and Elastic Network Interfaces to route traffic securely through the AWS network.

There are two types of VPC Endpoints:

  • Gateway Endpoints: For S3 and DynamoDB. Free and highly efficient.
  • Interface Endpoints: For all other services like EC2, IAM, STS, RDS, etc. Incur costs.

Interface endpoints create an elastic network interface in your subnet, allowing for private connectivity using private IP addresses. Gateway endpoints only support S3 and DynamoDB resources.

Endpoint Type Services Supported Cost Routing Mechanism
Gateway S3, DynamoDB Free Route table association
Interface EC2, IAM, STS, Secrets Manager, RDS, etc. Incur costs Elastic Network Interface with private IPs

Why VPC Endpoints Matter for Terraform in Air-Gapped Environments

When Terraform operates in an AWS environment isolated from the public internet, VPC Endpoints act as the backbone, allowing Terraform to interact privately with AWS services. This guide is at a beginner level difficulty.

Common Terraform operations that require private AWS connectivity include:

  • Storing Terraform state files in S3
  • Downloading modules or remote files from an S3 bucket
  • Using DynamoDB for state locking
  • Creating and managing EC2 instances, IAM roles, and secrets via the AWS provider

Without endpoints, terraform init and terraform plan will fail due to lack of internet access. With endpoints, Terraform can use standard AWS service URLs without modification when private DNS is enabled.

Gateway Endpoints for Terraform State

S3 Gateway Endpoint

The S3 Gateway Endpoint is required for storing Terraform state files in S3 and downloading modules or remote files from an S3 bucket.

Create the S3 Gateway Endpoint:

aws ec2 create-vpc-endpoint \ --vpc-id <vpc-id> \ --service-name com.amazonaws.<region>.s3 \ --route-table-ids <route-table-id>

Route Table: Add the endpoint to the route table associated with your private subnets.

Permissions: Restrict access to specific buckets using an endpoint policy:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": "s3:*", "Resource": [ "arn:aws:s3:::my-terraform-state-bucket", "arn:aws:s3:::my-terraform-state-bucket/*" ] } ] }

Terraform example for a Gateway endpoint:

module "vpce_endpoint" { source = "./vpc-endpoint" vpc_id = module.networking.dev_proj_1_vpc_id service_name = "com.amazonaws.eu-west-1.s3" vpc_endpoint_type = "Gateway" route_table_ids = [module.networking.dev_proj_1_private_route_table_ids] }

vpc-endpoint/main.tf

```
variable "vpcid" {}
variable "servicename" {}
variable "vpcendpointtype" {}
variable "routetableids" {}

resource "awsvpcendpoint" "s3" {
vpcid = var.vpcid
servicename = var.servicename
vpcendpointtype = var.vpcendpointtype
routetableids = var.routetableids
tags = {
Name = "dev-proj-1-vpce-s3"
}
}
```

DynamoDB Gateway Endpoint

The DynamoDB Gateway Endpoint is required if you are using DynamoDB for state locking.

Create the DynamoDB Gateway Endpoint:

aws ec2 create-vpc-endpoint \ --vpc-id <vpc-id> \ --service-name com.amazonaws.<region>.dynamodb \ --route-table-ids <route-table-id>

No additional endpoint policies are needed for DynamoDB.

Module example with both endpoints:

module "minimum_vpc_endpoints" { source = "boldlink/vpc-endpoints/aws/" version = "<latest_version_nr>" vpc_id = local.vpc_id tags = var.tags vpc_endpoints = [ { service_name = "com.amazonaws.${local.region}.dynamodb" vpc_endpoint_type = "Gateway" name = "DynamoDB" route_table_ids = flatten(local.route_table_ids) policy = data.aws_iam_policy_document.ddb_endpoint_policy.json } ] }

Interface Endpoints for Terraform-Managed AWS Services

Terraform needs interface VPC Endpoints for most AWS services. The key services and how to configure them follow.

  • ec2
  • iam
  • sts
  • secretsmanager

Create interface endpoints with subnet and security group IDs:

aws ec2 create-vpc-endpoint \ --vpc-id <vpc-id> \ --service-name com.amazonaws.<region>.ec2 \ --subnet-ids <subnet-id> \ --security-group-ids <security-group-id>

aws ec2 create-vpc-endpoint \ --vpc-id <vpc-id> \ --service-name com.amazonaws.<region>.iam \ --subnet-ids <subnet-id> \ --security-group-ids <security-group-id>

aws ec2 create-vpc-endpoint \ --vpc-id <vpc-id> \ --service-name com.amazonaws.<region>.sts \ --subnet-ids <subnet-id> \ --security-group-ids <security-group-id>

aws ec2 create-vpc-endpoint \ --vpc-id <vpc-id> \ --service-name com.amazonaws.<region>.secretsmanager \ --subnet-ids <subnet-id> \ --security-group-ids <security-group-id>

Security Groups for Interface Endpoints

For interface VPC Endpoints, ensure the associated security groups allow incoming traffic from the private subnet CIDRs.

Example Security Group Rules:

  • Allow HTTPS port 443 traffic from private subnet CIDRs.

The Terraform AWS VPC Endpoint Module is designed to create VPC endpoints on an existing VPC in your AWS infrastructure.

This module offers the ability to automatically generate a dedicated security group for all Interface endpoints when the createendpointsg variable is set to true, we recommend this setting to be set.

  • This module is easy to use with simplified examples
  • Removes the complexity of managing multiple resources manually
  • Deploys the needed resources faster

Examples available here.

You may notice a skipped Checkov alert CKV2AWS5 in the complete example of our Terraform module for VPC endpoints. This alert is intentionally skipped as our VPC endpoints are configured to utilize security groups only for specific Interface VPC endpoints. The security group attached is specifically designed to allow SSL/TLS inbound traffic on port 443.

Private DNS and Validation

Enable private DNS names so that Terraform can use the standard AWS service URLs e.g., ec2.amazonaws.com without modification.

aws ec2 modify-vpc-endpoint \ --vpc-endpoint-id <endpoint-id> \ --private-dns-enabled

Validating the Setup

After creating the necessary VPC Endpoints:

  • Run terraform init and ensure it can access the S3 bucket and DynamoDB table
  • Deploy a sample resource e.g., EC2 instance or IAM role to verify connectivity to the respective AWS services

After establishing a secure connection to the VPC endpoint, we can now access and list the S3 buckets.

Step-by-Step Configuration Summary

  • S3 Gateway Endpoint
  • DynamoDB Gateway Endpoint
  • Interface Endpoints for Terraform-Managed AWS Services
  • Configure Security Groups for VPC Endpoints
  • Enable Private DNS for Interface Endpoints

This guide walks you through setting up a VPC, creating subnets, configuring route tables, and finally creating VPC endpoints using Terraform.

Conclusion

Running Terraform in an air-gapped AWS environment requires thoughtful planning and VPC Endpoint configurations. By setting up the necessary endpoints S3, DynamoDB, EC2, IAM, etc., you enable Terraform to interact with AWS services securely without internet access.

This setup ensures a secure, compliant, and efficient workflow, even in the most restrictive environments.

All code in this post can be found on my GitHub.

Sources

  1. Running Terraform in an Air-Gapped Environment — Part 2
  2. TerraformFoundation terraform-aws-vpc-endpoints
  3. Creating and Linking VPC Endpoint of Type Interface Using Terraform and AWS Management Console
  4. Creating AWS VPC Endpoints Terraform Step by Step Guide

Related Posts