Terraform Provider Elasticsearch by Phillip Baker

The management of search and analytics infrastructure requires a precise orchestration layer to ensure that indices, watchers, and cluster configurations remain consistent across diverse environments. The terraform-provider-elasticsearch, developed by phillbaker, serves as a critical bridge between HashiCorp Terraform's Infrastructure as Code (IaC) paradigm and the Elasticsearch ecosystem. This provider allows operators to define their Elasticsearch resources—specifically focusing on compatibility across version 5, version 6, and version 7 of Elasticsearch—directly within Terraform configuration files. By treating the search cluster's internal state as code, organizations can avoid the risks associated with manual API calls or GUI-based configuration, ensuring that monitoring alerts and resource definitions are version-controlled and reproducible.

The utility of this provider is particularly pronounced when dealing with hybrid cloud deployments, such as AWS Elasticsearch domains, where IAM-based authentication and secure transport layers must be coordinated with the internal logic of the Elasticsearch engine. Because it is based on an original Pull Request to the main Terraform project, it inherits a design philosophy centered on stability and adherence to the Terraform Plugin SDK standards. This allows it to integrate seamlessly into larger DevOps pipelines, enabling the automated deployment of X-Pack watchers and other critical operational components.

Provider Architecture and Version Compatibility

The architecture of the phillbaker elasticsearch provider is designed to maintain a wide range of backward compatibility, acknowledging that many enterprise environments do not migrate Elasticsearch versions overnight.

The provider is explicitly compatible with the following Elasticsearch versions:

• Elasticsearch v5
• Elasticsearch v6
• Elasticsearch v7

From a Terraform SDK perspective, the provider has evolved to support modern Terraform workflows while maintaining paths for legacy systems. Version 2.x of the provider utilizes version 2.x of the Terraform Plugin SDK. This specific SDK version is a hard requirement for those using Terraform 0.12 and higher. For users still operating within the constraints of Terraform 0.11, the provider maintains 1.x releases to ensure that older infrastructure pipelines do not break.

This versioning strategy is critical because the jump from Terraform 0.11 to 0.12 introduced significant changes in how providers are handled and how state is managed. By providing distinct release branches, the developer ensures that the impact of the SDK upgrade does not force an immediate, risky migration of the underlying Elasticsearch infrastructure.

Installation and System Configuration

Installing the terraform-provider-elasticsearch requires a manual binary placement approach, as it may not be available through the standard public registry in all configurations.

To install the provider, the user must download the appropriate binary for their operating system and place it in a directory accessible by the Terraform process. Once the binary is positioned, the local Terraform configuration must be informed of the binary's location via the .terraformrc file.

The required configuration for the ~/.terraformrc file is as follows:

hcl providers { elasticsearch = "/path/to/terraform-provider-elasticsearch" }

This configuration informs the Terraform CLI that whenever the elasticsearch provider is declared in a .tf file, it should bypass the remote registry and execute the local binary located at the specified absolute path. This is an essential step for ensuring that the correct version of the provider is used, especially in air-gapped environments or specialized development setups.

Provider Configuration and Authentication

The provider "elasticsearch" block is the central point of configuration. It defines how Terraform connects to the Elasticsearch cluster and how it authenticates its requests.

The configuration options are comprehensive, supporting both standard HTTP/HTTPS connections and integrated AWS authentication.

The following table details the available configuration parameters:

For a typical AWS deployment, the provider block would look like this:

hcl provider "elasticsearch" { url = "https://search-foo-bar-pqrhr4w3u4dzervg41frow4mmy.us-east-1.es.amazonaws.com" aws_access_key = "" aws_secret_key = "" aws_token = "" insecure = true cacert_file = "/path/to/ca.crt" sign_aws_requests = true }

The sign_aws_requests parameter is particularly important for security-conscious organizations. When enabled, Terraform signs the HTTP requests using the AWS Signature Version 4 process, allowing the Elasticsearch cluster to validate the identity of the Terraform runner via IAM roles rather than relying on static usernames and passwords.

X-Pack Watcher Implementation

One of the most powerful features of the phillbaker provider is its ability to manage elasticsearch_xpack_watch resources. X-Pack Watchers allow users to create alerting systems that trigger actions based on the state of the cluster or the data within the indices.

Monitoring Cluster Health

The provider can be used to define a watcher that monitors the health status of the Elasticsearch cluster. If the cluster health drops to a "red" state—indicating that at least one primary shard is not allocated—the watcher can trigger an external notification via Slack.

The following configuration demonstrates a "Cluster Health Red" watcher:

hcl resource "elasticsearch_xpack_watch" "cluster-health-red" { watch_id = "cluster-health-red" body = <<EOF { "trigger": { "schedule": { "interval": "10m" } }, "input": { "http": { "request": { "scheme": "http", "host": "localhost", "port": 9200, "method": "get", "path": "/_cluster/health", "headers": { "Authorization": "Basic ${base64encode('username:password')}" } } } }, "condition": { "compare": { "ctx.payload.status": { "eq": "red" } } }, "actions": { "notify-slack": { "throttle_period_in_millis": 300000, "slack": { "account": "monitoring", "message": { "from": "watcher", "to": [ "#my-slack-channel" ], "text": "Elasticsearch Monitoring", "attachments": [ { "color": "danger", "title": "Cluster Health Warning - RED", "text": "elasticsearch cluster health is RED" } ] } } } }, "metadata": { "xpack": { "type": "json" }, "name": "Cluster Health Red" } } EOF }

In this example, the throttle_period_in_millis is set to 300000 (5 minutes), which prevents the Slack channel from being flooded with alerts if the cluster remains red for an extended period. The base64encode function is used within the header to handle Basic Authentication, ensuring that the credentials are passed correctly to the Elasticsearch API.

Monitoring JVM Memory Pressure

Another critical operational metric is the JVM heap usage. High JVM memory pressure can lead to frequent garbage collection cycles, increased latency, or even cluster instability. The provider allows for the creation of a watcher that uses the Painless scripting language to evaluate memory usage across all nodes.

The configuration for JVM memory monitoring is as follows:

hcl resource "elasticsearch_xpack_watch" "jvm-memory-usage" { watch_id = "jvm-memory-usage" body = <<EOF { "trigger": { "schedule": { "interval": "10m" } }, "input": { "http": { "request": { "scheme": "http", "host": "localhost", "port": 9200, "method": "get", "path": "/_nodes/stats/jvm", "params": { "filter_path": "nodes.*.jvm.mem.heap_used_percent" }, "headers": {} } } }, "condition": { "script": { "lang": "painless", "source": "ctx.payload.nodes.values().stream().anyMatch(node -> node.jvm.mem.heap_used_percent > 75)" } }, "actions": { "notify-slack": { "throttle_period_in_millis": 600000, "slack": { "account": "monitoring", "message": { "from": "watcher", "to": [ "#my-slack-channel" ], "text": "Elasticsearch Monitoring", "attachments": [ { "color": "danger", "title": "JVM Memory Pressure Warning", "text": "JVM Memory Pressure has been > 75% on one or more nodes for the last 5 minutes." } ] } } } }, "metadata": { "xpack": { "type": "json" }, "name": "JVM Memory Pressure Warning" } } EOF }

This specific implementation uses a painless script to stream through the payload of all nodes and check if any single node has a heap_used_percent greater than 75%. This is a proactive monitoring approach that allows administrators to scale their cluster or optimize their queries before a crash occurs.

Development and Contribution Workflow

For developers looking to extend the functionality of the provider or fix bugs, a specific local development environment is required.

System Requirements

To build the provider from source, the developer must have the following installed:

• Golang version 1.11 or higher

Build Process

The provider can be compiled into a binary using the standard Go toolchain. The following command is used to build the executable to a specific destination:

bash go build -o /path/to/binary/terraform-provider-elasticsearch

Local Debugging and Testing

Debugging a Terraform provider can be challenging because the provider runs as a separate process managed by the Terraform CLI. To solve this, the provider supports a debuggable mode.

First, the developer must ensure that the test and debug profiles have the following environmental variables set:

• ELASTICSEARCH_URL=http://localhost:9200_
• TF_ACC=1 (This enables Acceptance Tests)

To start the provider in debug mode, the following commands are executed:

bash go build ./terraform-provider-elasticsearch -debuggable

When started with the -debuggable flag, the provider prints its plugin address to the terminal. An example output looks like this:

json {"@level":"debug","@message":"plugin address","@timestamp":"2022-05-17T10:10:04.331668+01:00","address":"/var/folders/32/3mbbgs9x0r5bf991ltrl3p280000gs/T/plugin1346340234","network":"unix"}

To connect a running Terraform instance to this debuggable process, the developer must set the TF_REATTACH_PROVIDERS environmental variable. The value of this variable is a JSON string containing the protocol, PID, and socket address of the running provider.

Example of the TF_REATTACH_PROVIDERS variable:

bash export TF_REATTACH_PROVIDERS='{"registry.terraform.io/phillbaker/elasticsearch":{"Protocol":"grpc","ProtocolVersion":5,"Pid":79075,"Test":true,"Addr":{"Network":"unix","String":"/var/folders/32/3mbbgs9x0r5bf991ltrl3p280000gs/T/plugin1346340234"}}}'

Once this variable is exported in a separate terminal, the developer can run Terraform commands:

bash cd <my-project/terraform> terraform apply

Terraform will then use the local, debuggable provider instead of downloading a binary, allowing the developer to set breakpoints in their IDE and inspect the execution flow in real-time.

Contributing to the Project

The project follows a standard open-source contribution workflow via GitHub.

The steps for contributing are as follows:

• Fork the repository at https://github.com/phillbaker/terraform-provider-elasticsearch/fork
• Create a feature branch using the command: git checkout -b my-new-feature
• Commit changes with descriptive messages: git commit -am 'Add some feature'
• Push the branch to the origin: git push origin my-new-feature
• Create a new Pull Request for review.

Operational Analysis and Conclusion

The phillbaker terraform-provider-elasticsearch fills a critical gap in the Elasticsearch operational toolkit by bringing the rigor of Infrastructure as Code to the internal configuration of the search engine. While many providers focus on the "outer" shell of the service (such as creating the AWS domain itself), this provider focuses on the "inner" shell—the indices, settings, and X-Pack watchers that define how the search engine actually behaves.

The implementation of the elasticsearch_xpack_watch resource is the standout feature. By allowing the definition of complex JSON payloads via Terraform heredocs (<<EOF), the provider enables "Monitoring as Code." This means that a company's alerting thresholds (e.g., the 75% JVM heap threshold) are no longer hidden in a UI or scattered across various API calls; they are documented and versioned in the same repository as the infrastructure.

From a technical standpoint, the provider's reliance on the Terraform Plugin SDK v2.x ensures compatibility with the modern Terraform 0.12+ ecosystem, while the provision of 1.x releases shows a commitment to supporting legacy environments. The ability to handle AWS-specific signing requirements via sign_aws_requests further emphasizes its readiness for enterprise cloud environments.

Ultimately, the use of this provider reduces the operational risk of "configuration drift" within an Elasticsearch cluster. When monitoring alerts are managed through Terraform, the state of the cluster's alerting system is guaranteed to match the desired state defined in the code. This creates a more resilient search infrastructure capable of self-documenting its health checks and alerting logic.