Amazon Simple Queue Service is a fully managed message queuing service that helps decouple and scale distributed applications. Using Terraform to provision and manage SQS queues ensures consistent, automated, and repeatable infrastructure deployments. This article covers defining and configuring Amazon SQS queues using Terraform with example configurations that streamline message handling in cloud architecture.
What Is an SQS Queue
Amazon SQS is a fully managed message queuing service that enables decoupling of distributed systems by allowing asynchronous communication between microservices and other components.
SQS works by temporarily storing messages in a queue until they are retrieved and processed by a consuming service. This improves system resilience and scalability by isolating producers from consumers. Producers send messages to the queue URL, and consumers poll it.
Amazon SQS is the easiest way to decouple services on AWS. Need to process work asynchronously? Throw a message on a queue and let a worker pick it up. Need to absorb traffic spikes? Queue the requests and process them at your own pace. SQS handles the complexity of distributed messaging so you don't have to.
In this post, we will create SQS queues with Terraform, covering standard and FIFO queues, dead-letter queues, encryption, and the access policies that tie everything together.
Standard vs FIFO Queue Characteristics
Standard Queue
A standard queue provides at-least-once delivery with best-effort ordering. It's the default and handles the vast majority of use cases.
FIFO Queue
In Terraform, you model this queue as code using the awssqsqueue resource.
The key points are:
- The queue name must end with .fifo
- You must enable fifoqueue = true
- For higher throughput you typically enable contentbased_deduplication
| Property | Standard Queue | FIFO Queue |
|---|---|---|
| Ordering | Best effort | Strictly ordered |
| Delivery guarantee | At least once | Exactly once processing |
| Name requirement | Any name | Must end with .fifo |
| Throughput | High | Lower, up to 300 messages per second |
| Deduplication | Not applicable | contentbaseddeduplication or message deduplication ID |
| Use case | Decoupling and scaling | Order critical processing |
Provider Configuration and Basic Resource
The provider block configures the authentication details and default settings for interacting with AWS.
hcl
provider "aws" {
region = "us-east-1" # Specify your desired AWS region
}
Create SQS topic
Define the SQS topic resource in your Terraform configuration file, specifying the name for the topic.
hcl
resource "aws_sqs_queue" "my_queue" {
name = "my-sqs-topic"
}
Step 4: Now Initialize Terraform And Execute Terraform Commands
Now initialize terraform by using following command
hcl
terraform init
Now execute terraform execution commands by using following commands
hcl
terraform fmt
terraform validate
terraform plan
Now execute terraform apply command by using following command
hcl
terraform apply --auto-approve
Production Queue With Custom Behavior
An AWS SQS queue in Terraform is just a resource block that describes how your queue should behave. In a production environment, you usually want:
- A main queue for normal messages
- A dead letter queue for messages that keep failing
- Sensible timeouts and retention settings
The configuration below shows a common production pattern for a Terraform SQS queue:
```hcl
provider "aws" {
region = "eu-central-1"
}
Dead letter queue
resource "awssqsqueue" "ordersdlq" {
name = "orders-dlq"
messageretention_seconds = 1209600 # 14 days
tags = {
Environment = "production"
Service = "orders"
}
}
Main production queue
resource "awssqsqueue" "ordersqueue" {
name = "orders-queue"
# How long a worker has to process a message
visibilitytimeoutseconds = 60
# Keep messages for 4 days
messageretentionseconds = 345600
# Optional delivery delay for new messages
delayseconds = 0
# Long polling to reduce empty receives
receivewaittimeseconds = 20
# Move failing messages to the DLQ after 5 failed receives
redrivepolicy = jsonencode({
deadLetterTargetArn = awssqsqueue.orders_dlq.arn
maxReceiveCount = 5
})
tags = {
Environment = "production"
Service = "orders"
}
}
```
The orders_queue is your main production queue where the application publishes messages.
This creates a standard SQS queue with reasonable defaults:
```hcl
Standard SQS queue
resource "awssqsqueue" "orders" {
name = "order-processing"
# How long a consumer has to process a message before it becomes
# visible again (in seconds)
visibilitytimeoutseconds = 60
# How long messages stay in the queue if not consumed (in seconds)
# Maximum is 14 days (1209600)
messageretentionseconds = 86400 # 1 day
# Maximum message size in bytes (max 1 MiB)
maxmessagesize = 1048576
# Long polling - wait up to 20 seconds for messages
# This reduces empty responses and API costs
receivewaittimeseconds = 20
# Delay delivery of new messages (0-900 seconds)
delayseconds = 0
tags = {
Environment = "production"
ManagedBy = "terraform"
}
}
```
The visibilitytimeoutseconds setting is critical. Set it to at least as long as your longest expected processing time.
FIFO Queue Example
```hcl
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
region = "us-east-1"
}
resource "awssqsqueue" "ordersfifo" {
name = "orders-events.fifo"
fifoqueue = true
contentbaseddeduplication = true
visibilitytimeoutseconds = 30
messageretentionseconds = 345600 # 4 days
delayseconds = 0
maxmessagesize = 262144
redrivepolicy = jsonencode({
deadLetterTargetArn = awssqsqueue.orders_dlq.arn
maxReceiveCount = 5
})
}
resource "awssqsqueue" "ordersdlq" {
name = "orders-events-dlq.fifo"
fifoqueue = true
contentbaseddeduplication = true
}
```
This configuration does the following in a straightforward way
- Creates a main FIFO queue called orders-events.fifo that guarantees ordered message delivery
- Enables content based deduplication so SQS uses the message body to detect duplicates within the 5 minute deduplication window
- Sets a visibility timeout of 30 seconds which gives consumers time to process each message before it can be received again
- Sets a retention period of four days for unconsumed messages
- Configures a FIFO dead letter queue and a redrive policy so messages that fail more than five times are
Encryption and Permissions
Use awssqsqueue_policy when you need cross-account access or access from AWS services that require a queue policy for example, SNS fan-out to SQS.
Keep these policies minimal and explicit, granting access only to the required principals and actions.
Align permissions with encryption
If you use a customer managed KMS key for queue encryption, make sure:
- The IAM role that accesses the queue also has kms:Encrypt, kms:Decrypt, and kms:GenerateDataKey* on the relevant KMS key
- The KMS key policy allows the SQS service and your application roles to use the key
If you stay with the default AWS-managed key for SQS encryption, AWS handles most of the key policy for you, and you usually do not need extra KMS permissions.
Managing both queue and KMS permissions in Terraform keeps your security posture consistent and avoids subtle runtime failures when messages are encrypted.
Example 2: Secure encrypted queue for sensitive data
In this article, we will show you how to define and configure an Amazon SQS queue using Terraform and provide example configurations that streamline message handling in your cloud architecture.
What we will cover:
- What is an SQS Queue?
- How to use SQS Queue in Terraform
- Example 1: Production queue with custom behavior
- Example 2: Secure encrypted queue for sensitive data
- Example of a FIFO Queue
- How to import existing AWS SQS queues into Terraform state
- Best practices for naming and permissions of SQS queues in Terraform
Importing Existing SQS Queues Into Terraform
Define the Terraform resource
Next, add an awssqsqueue resource to your Terraform configuration that represents the existing queue. Start with a minimal block:
hcl
resource "aws_sqs_queue" "orders_queue" {
name = "orders-queue"
# Optional: fill in settings to match the existing queue
# visibility_timeout_seconds = 60
# message_retention_seconds = 345600
# ...
}
The resource name orders_queue in this example is internal to Terraform. The name argument must match the real SQS queue name.
After importing, you can run terraform state show awssqsqueue.orders_queue to see the full set of arguments and align your configuration. Only add arguments in your HCL that you actually want Terraform to manage and leave computed attributes out of the configuration.
- Run the import
Once the resource exists in your .tf files, run terraform init if you haven't already, then import the queue:
hcl
terraform import aws_sqs_queue.orders_queue https://sqs.eu-central-1.amazonaws.com/123456789012/orders-queue
This tells Terraform:
- Use the resource awssqsqueue.orders_queue in the config,
- Link it to the existing SQS queue with that URL.
If you use a newer Terraform version you can also define an import block in your configuration instead of running terraform import, but the command shown here is still supported and simple to use.
After the import completes run terraform plan to see any differences between your configuration and the actual queue.
Best Practices for Naming and Permissions
Best practices for naming and permissions of SQS queues in Terraform include consistent tagging, explicit visibility timeouts, dead letter queue integration, and minimal queue policies.
| Practice | Recommendation |
|---|---|
| Naming | Use descriptive names with environment prefix |
| Tags | Environment, Service, ManagedBy |
| Visibility timeout | Match longest processing time |
| Message retention | 1 to 14 days depending on SLA |
| Redrive policy | Always configure DLQ with maxReceiveCount |
| Long polling | Set receivewaittime_seconds to 20 |
| Encryption | Prefer customer managed KMS for sensitive data |
Managing Terraform Resources With Spacelift
Terraform is really powerful, but to achieve an end-to-end secure GitOps approach, you need to use a product that can run your Terraform workflows.
How to manage Terraform resources with Spacelift
Terraform is really powerful, but to achieve an end-to-end secure GitOps approach, you need to use a product that can run your Terraform workflows.
Conclusion
Amazon Simple Queue Service remains as an essential part inside the AWS environment, offering a scalable, reliable, and completely managed message queuing service for dispersed systems. All through this conversation, we've explored SQS's capacity to work with asynchronous correspondence between various pieces of an application, empowering decoupling and consistent scaling of parts. By utilizing SQS, developers can ensure message reliability and availability, even notwithstanding system failures or spikes traffic.
Using Terraform to provision and manage SQS queues ensures consistent, automated, and repeatable infrastructure deployments. Standard queues provide at-least-once delivery for high-throughput decoupling, while FIFO queues guarantee strict ordering for order-sensitive workflows. Dead letter queues with redrive policies protect against poison messages, visibility timeouts align with processing durations, and long polling reduces API costs. Importing existing queues preserves drift-free state, and aligning queue and KMS permissions in Terraform prevents runtime encryption failures. Together these patterns allow production-ready message handling that scales with demand while maintaining reliability and security.