Terraform kubectl Provider YAML Manifest Orchestration for Kubernetes

The terraform-provider-kubectl project by gavinbunney occupies a specific niche in the Terraform ecosystem by translating the native YAML expression of Kubernetes into a first-class Terraform lifecycle. The provider enables management of Kubernetes resources using raw YAML manifests within Terraform's declarative infrastructure-as-code workflow. This bridging function connects Terraform's resource lifecycle management with Kubernetes' native YAML-based resource definitions, allowing users to apply, update, and delete Kubernetes resources while maintaining Terraform state tracking and drift detection. The positioning of the provider is explicitly as a bridge between two declarative models that historically required separate toolchains, kubectl for imperative YAML application and Terraform for stateful infrastructure graph management.

The core value proposition is that Kubernetes resources can be expressed in the format Kubernetes loves best, yaml, and still participate in Terraform plan, apply, and state operations. The provider's architecture is documented as covering architecture, core components, and development infrastructure, with references to separate documentation for Provider Configuration and for the kubectl_manifest Resource. The provider is described as the best way of managing Kubernetes resources in Terraform by allowing yaml usage, and it has been used by many large Kubernetes installations to completely manage the lifecycle of Kubernetes resources.

Provider Architecture and Core Purpose

The terraform-provider-kubectl provider is a Terraform provider that enables management of Kubernetes resources using raw YAML manifests within Terraform's declarative infrastructure-as-code workflow. This provider bridges the gap between Terraform's resource lifecycle management and Kubernetes' native YAML-based resource definitions, allowing users to apply, update, and delete Kubernetes resources while maintaining Terraform state tracking and drift detection.

The impact for operators is that YAML authored for Kubernetes does not need to be duplicated into Terraform HCL resource schemas. Teams can keep manifests as the source of truth and still receive Terraform's state consistency guarantees. Drift detection ensures that manual changes made via kubectl or the API server are surfaced during Terraform plan, which closes a common governance gap in GitOps workflows where out-of-band edits go unnoticed.

Contextually, the provider serves as a robust bridge between Terraform's infrastructure management capabilities and Kubernetes' declarative resource model, enabling users to leverage YAML-based Kubernetes configurations within Terraform workflows while maintaining state consistency and drift detection. This bridging role is complementary to the HashiCorp maintained kubernetes provider, which interacts with resources supported by Kubernetes through native Terraform resources. The tutorial for that provider focuses on scheduling and exposing a NGINX deployment on a Kubernetes cluster and managing custom resources using Terraform, highlighting benefits such as Unified Workflow, Full Lifecycle Management, and Graph of Relationships.

Unified Workflow means that if an organization is already provisioning Kubernetes clusters with Terraform, the same configuration language can be used to deploy applications into the cluster. Full Lifecycle Management means Terraform doesn't only create resources, it updates, and deletes tracked resources without requiring inspection of the API to identify those resources. Graph of Relationships means Terraform understands dependency relationships between resources, for example if a Persistent Volume Claim claims space from a particular Persistent Volume, Terraform won't attempt to create the claim if it fails to create the volume.

kubectl_manifest Resource and Drift Detection

The core of this provider is the kubectl_manifest resource, allowing free-form yaml to be processed and applied against Kubernetes. This yaml object is then tracked and handles creation, updates and deleted seamlessly, including drift detection.

The direct fact is that a single resource type accepts arbitrary YAML. The impact is that operators can import existing manifests, Helm-rendered outputs, or Kustomize-generated YAML without rewriting them into Terraform provider-specific blocks. The seamless handling of creation, updates and deletion reduces toil associated with manual kubectl apply -f cycles.

Contextually, drift detection is the key differentiator from running kubectl in a CI job. Terraform state is used to record the last known applied configuration, and the provider compares the live cluster object with the declared YAML. When the API server reports a difference, Terraform plan reports it as drift, allowing remediation via apply.

A usage pattern example from the reference material shows a provider block and a manifest resource:

provider "kubectl" { host = var.eks_cluster_endpoint cluster_ca_certificate = base64decode(var.eks_cluster_ca) token = data.aws_eks_cluster_auth.main.token load_config_file = false }

resource "kubectl_manifest" "test" { yaml_body = <<YAML apiVersion: couchbase.com/v1 kind: CouchbaseCluster metadata: name: name-here-cluster spec: baseImage: name-here-image version: name-here-image-version authSecret: name-here-operator-secret-name exposeAdminConsole: true adminConsoleServices: - data cluster: dataServiceMemoryQuota: 256 indexServiceMemoryQuota: 256 searchServiceMemoryQuota: 256 eventingServiceMemoryQuota: 256 analyticsServiceMemoryQuota: 1024 indexStorageSetting: memory_optimized autoFailoverTimeout: 120 autoFailoverMaxCount: 3 autoFailoverOnDataDiskIssues: true autoFailoverOnDataDiskIssuesTimePeriod: 120 autoFailoverServerGroup: false YAML }

The example demonstrates that custom resources such as CouchbaseCluster can be managed via the kubectl_manifest resource. The YAML is embedded directly in the HCL configuration, and Terraform will apply it through the kubectl client library.

Data Resources for YAML Directory Processing and Templating

This provider is described as having a set of helpful data resources to process directories of yaml files and inline templating is available. The direct fact is that data resources exist to handle collections of manifests.

The impact for teams is that large directories of YAML, often produced by operators or CI pipelines, can be ingested without maintaining a one-to-one resource block per file. Inline templating support means variables can be interpolated before application, allowing reuse of manifest templates across environments.

Contextually, this capability sits alongside the core kubectlmanifest resource. While kubectlmanifest handles a single YAML object, the data resources extend the provider to handle directory processing, which is essential for platform teams managing hundreds of manifests in a single Terraform run.

Installation Paths and Version Constraints

Installation is supported for Terraform 0.13+. The provider can be installed and managed automatically by Terraform. Sample versions.tf file is referenced as an option for automatic management. If automatic management is not desired, a binary can be downloaded for the system from the release page, then either placed at the root of the Terraform folder or in the Terraform plugin folder on the system. User Guide details are referenced for installation and all provided data and resource types.

The impact of automatic installation is that Terraform will fetch the correct provider version based on the required_providers block, reducing manual binary management. The alternative manual path gives control over the exact binary and placement, useful in air-gapped environments.

Contextually, the provider requires Terraform 0.13+ which aligns with the provider protocol introduced in that version. The binary placement options reflect Terraform's plugin discovery mechanism where a binary at the module root or in the user plugin directory is discovered automatically.

A development note from the repository shows a shell snippet for building and preparing the binary:

. && \ chmod +x terraform-provider-kubectl* && \ rm -rf terraform-provider-kubectl-tmp && \ rm -rf terraform-provider-kubectl.zip && \ popd

The snippet illustrates the post-build steps used in CI to make the binary executable and clean temporary artifacts.

Provider Configuration and Retry Logic

The provider implements configurable retry logic through the kubectlApplyRetryCount variable, allowing resilient operations against unstable clusters:

The provider argument is applyretrycount. The environment variable is KUBECTLPROVIDERAPPLYRETRYCOUNT. Sources reference kubernetes/provider.go238-255.

The direct fact is that retry count is configurable. The impact is that transient API server errors, such as 500s or connection resets during apply, will be retried a configurable number of times before failing the Terraform operation. This resilience is critical for clusters under load or with network instability.

Contextually, retry logic complements Terraform's own retry mechanisms and the Kubernetes client-go exponential backoff. By exposing a provider-level knob, operators can tune for cluster specific characteristics without modifying Terraform core behavior.

The provider uses Kubernetes discovery mechanisms to dynamically determine available API resources and their REST mappings. This dynamic discovery avoids hardcoding API groups and versions, allowing the provider to work with CRDs that may be installed after Terraform configuration is written.

Go Module Dependencies and Integration Stack

Key Go modules for Kubernetes integration and Terraform provider functionality are listed in go.mod.

Sources: go.mod7-29

The direct fact is the specific module versions. The impact is that the provider builds on terraform-plugin-sdk/v2 for provider lifecycle, uses client-go for API communication, uses k8s.io/kubectl and cli-runtime for kubectl-style apply semantics, and includes Kustomize API for integration scenarios.

Contextually, the combination of kubectl and client-go allows the provider to both apply manifests like kubectl apply and perform state queries via the Go client. Kustomize API support suggests the provider can process Kustomize overlays without invoking an external kustomize binary.

Development Workflow and Build Requirements

If you wish to work on the provider, you'll first need Go installed on the machine, version 1.12+ is required. You'll also need to correctly setup a GOPATH, as well as adding $GOPATH/bin to $PATH. To compile the provider, run make build.

The direct fact is the minimum Go version and build command. The impact is that contributors can build a local binary for testing without publishing to a registry. The requirement to setup GOPATH reflects the provider's historical Go workspace expectations.

Contextually, the development guide aligns with standard Terraform provider development practices where a Makefile wraps go build with appropriate tags and output naming. The Go version requirement of 1.12+ is a minimum, and the current module versions indicate a much newer toolchain is used in practice.

Usage Pattern Examples

The provider configuration can be expressed for EKS or other clusters using the kubectl provider block with host, clustercacertificate, token, and loadconfigfile settings. The example shows:

provider "kubectl" { host = var.eks_cluster_endpoint cluster_ca_certificate = base64decode(var.eks_cluster_ca) token = data.aws_eks_cluster_auth.main.token load_config_file = false }

This pattern isolates authentication from manifest content. The impact is that Terraform can manage Kubernetes resources without requiring a local kubeconfig file, allowing CI/CD pipelines to authenticate via cloud provider data sources.

The tutorial for the HashiCorp kubernetes provider provides a contrasting pattern for native Terraform resources:

terraform { required_providers { kubernetes = { source = "hashicorp/kubernetes" version = "~> 3.0" } } }

Variables for host, clientcertificate, clientkey, and clustercacertificate are defined, and the provider block decodes base64 certificates.

provider "kubernetes" { host = var.host client_certificate = base64decode(var.client_certificate) client_key = base64decode(var.client_key) cluster_ca_certificate = base64decode(var.cluster_ca_certificate) }

The tutorial assumes some basic familiarity with Kubernetes and kubectl, and assumes familiarity with the usual Terraform plan/apply workflow. It creates a directory named learn-terraform-deploy-nginx-kubernetes.

mkdir learn-terraform-deploy-nginx-kubernetes

cd learn-terraform-deploy-nginx-kubernetes

Then it creates a file named kubernetes.tf and adds configuration. The cloud provider tabs will configure the Kubernetes provider using cloud-specific auth tokens.

The impact of the HashiCorp provider tutorial is to illustrate the alternative native resource model. The kubectl provider complements this by allowing YAML-first workflows while the kubernetes provider requires HCL resource definitions.

Relationship to HashiCorp Kubernetes Provider

The reference material includes the HashiCorp kubernetes provider tutorial which emphasizes Unified Workflow, Full Lifecycle Management, and Graph of Relationships. The kubectl provider shares the Unified Workflow benefit but extends it to YAML manifests. Full Lifecycle Management is provided via Terraform state and drift detection in both providers. Graph of Relationships is native to Terraform's dependency graph in both cases, but the kubectl provider's free-form YAML means dependencies must be expressed via Terraform depends_on or implicit ordering rather than native resource attributes.

Contextually, organizations often use both providers together: the kubernetes provider to create namespaces or service accounts, and the kubectl provider to apply complex manifests that include custom resources not yet covered by the native provider.

Conclusion

The terraform-provider-kubectl project delivers a focused bridge between Terraform state management and Kubernetes YAML application. By centering on the kubectl_manifest resource, configurable retry logic, dynamic API discovery, and Go module integrations including terraform-plugin-sdk/v2, client-go, kubectl, cli-runtime, and Kustomize API, the provider allows teams to maintain YAML as the source of truth while gaining Terraform's plan, apply, drift detection, and state tracking. Installation supports Terraform 0.13+ with automatic provider management or manual binary placement. Development requires Go 1.12+ and a standard GOPATH setup with make build for compilation. The provider coexists with the HashiCorp kubernetes provider, offering a YAML-first alternative for manifest management alongside native Terraform resource definitions. The combination of data resources for directory processing and templating, retry resilience, and discovery-based API handling makes the provider suitable for large Kubernetes installations that require complete lifecycle management of Kubernetes resources within Terraform workflows.

Sources

1. terraform-provider-kubectl DeepWiki
2. terraform-provider-kubectl pkg.go.dev
3. terraform-provider-kubectl GitHub
4. HashiCorp Terraform Kubernetes Tutorial