The terraform-provider-kubectl project represents a deliberate architectural bridge between Terraform’s infrastructure-as-code resource model and Kubernetes’ native YAML-based resource definitions. The provider enables declarative management of Kubernetes resources through native YAML manifests and functions as a conduit that allows Terraform to apply, manage, and track the lifecycle of any Kubernetes resource directly through Terraform state. The documentation describes the overall system architecture, core components, and key integration points as a central reference for how the provider operates between Terraform configuration and the Kubernetes API server. Specific usage instructions and configuration details are positioned as a separate Getting Started track, while the core architectural description remains focused on the provider’s role as a YAML-centric intermediary.
The provider’s fundamental value proposition is the ability to manage Kubernetes resources in Terraform by allowing Terraform to use the artifact Kubernetes loves best, yaml. The core of this provider is the kubectl_manifest resource, which allows free-form yaml to be processed and applied against Kubernetes. This yaml object is then tracked and handles creation, updates and deletion seamlessly, including drift detection. A set of helpful data resources to process directories of yaml files and inline templating is available. The provider has been used by many large Kubernetes installations to completely manage the lifecycle of Kubernetes resources. The provider can be installed and managed automatically by Terraform.
Architecture and Core Components
The provider operates as a bridge between Terraform’s resource management lifecycle and Kubernetes’ API server, with YAML processing at its core. The provider’s architecture centers around several key components that handle the complete lifecycle from Terraform configuration to Kubernetes resource management.
The provider implements a sophisticated YAML processing system that handles templating, multi-document parsing, and manifest validation before applying resources to Kubernetes. This processing layer is the point where raw Terraform configuration strings are transformed into Kubernetes API objects that can be submitted to the API server. The impact for operators is that manifests can be authored in familiar Kubernetes YAML syntax rather than being translated into provider-specific schema blocks, which reduces cognitive overhead and preserves the portability of the manifests across tooling.
The provider uses dynamic Kubernetes clients to discover API resources and manage the complete lifecycle of resources, including creation, updates, deletion, and rollout monitoring. Dynamic client discovery means the provider does not require a static schema for every Kubernetes API group and version. The consequence for users is that custom resources and new API objects become manageable without waiting for a provider schema update. Rollout monitoring is integrated into the lifecycle so that Terraform can wait for readiness signals before considering an operation complete.
The provider includes comprehensive testing, build automation, and quality assurance systems that ensure reliability across multiple Kubernetes and Terraform versions. The testing and quality systems are designed to maintain compatibility as both Terraform and Kubernetes evolve.
| Component | Purpose | Key Files |
|---|---|---|
| Build System | Compilation and distribution | main.go, go.mod, Makefile |
| Testing Framework | Unit and acceptance tests | kubernetes/provider_test.go, _examples/ |
| CI/CD Pipeline | Automated testing and releases | .github/workflows/, scripts/ |
| Code Quality | Linting, formatting, validation | gofmt, go vet, errcheck |
| Version Matrix | Multi-version compatibility testing | K8s versions, Terraform versions |
The table reflects the internal organization used to sustain the provider. The Build System handles compilation and distribution through main.go, go.mod, and Makefile. The Testing Framework holds unit and acceptance tests in kubernetes/provider_test.go and _examples/. The CI/CD Pipeline automates testing and releases using .github/workflows/ and scripts/. Code Quality relies on gofmt, go vet, and errcheck. The Version Matrix manages multi-version compatibility testing across K8s versions and Terraform versions.
The provider supports the last 7 Kubernetes releases and last 4 stable Terraform versions, with automated testing. This support window directly affects adoption risk because users operating on recent Kubernetes clusters can rely on continued compatibility without manual fork maintenance.
The kubectl_manifest Resource and Drift Detection
The kubectl_manifest resource is the central resource type. It accepts free-form yaml and processes it against Kubernetes. Once a manifest is registered, the provider tracks it through Terraform state and handles creation, updates and deletion seamlessly.
Drift detection is an integral part of tracking. When Kubernetes resources are modified outside Terraform, the provider can detect divergence between the desired YAML manifest and the live API object. The impact of drift detection is that operators are alerted to configuration changes made via kubectl or other tools, preserving the declarative intent of Terraform and preventing configuration sprawl.
The resource also handles deletion. When a kubectl_manifest resource is removed from configuration, the corresponding Kubernetes object is removed according to Terraform’s destroy lifecycle. This ensures that Terraform remains the source of truth for the managed resources.
An example configuration pattern shows the resource in use with a provider block and a manifest.
provider "kubectl" {
host = var.eks_cluster_endpoint
cluster_ca_certificate = base64decode(var.eks_cluster_ca)
token = data.aws_eks_cluster_auth.main.token
load_config_file = false
}
resource "kubectl_manifest" "test" {
yaml_body = <<YAML
apiVersion: couchbase.com/v1
kind: CouchbaseCluster
metadata:
name: name-here-cluster
spec:
baseImage: name-here-image
version: name-here-image-version
authSecret: name-here-operator-secret-name
exposeAdminConsole: true
adminConsoleServices:
- data
cluster:
dataServiceMemoryQuota: 256
indexServiceMemoryQuota: 256
searchServiceMemoryQuota: 256
eventingServiceMemoryQuota: 256
analyticsServiceMemoryQuota: 1024
indexStorageSetting: memory_optimized
autoFailoverTimeout: 120
autoFailoverMaxCount: 3
autoFailoverOnDataDiskIssues: true
autoFailoverOnDataDiskIssuesTimePeriod: 120
autoFailoverServerGroup: false
YAML
}
The example demonstrates a provider configured with host, clustercacertificate, token, and loadconfigfile false, followed by a kubectlmanifest resource with a yamlbody containing a CouchbaseCluster custom resource. The use of heredoc YAML allows multi-document manifests to be embedded directly in Terraform configuration.
YAML Processing and Data Resources
Beyond the single manifest resource, the provider offers a set of helpful data resources to process directories of yaml files and inline templating. Directory processing allows an entire folder of Kubernetes manifests to be ingested as a single logical unit, which simplifies bulk operations such as applying Helm-unrendered charts or GitOps repositories.
Inline templating provides a mechanism to inject Terraform variables into YAML before submission. This preserves the native YAML structure while allowing Terraform interpolation, a compromise between pure YAML fidelity and Terraform parameterization.
Multi-document parsing is handled by the YAML processing system. Users can supply manifests containing multiple --- separated documents and the provider will correctly split and apply each document in dependency-aware order.
Manifest validation occurs prior to API submission, which reduces failed apply attempts by catching schema errors early. Validation is contextual because it relies on the dynamic Kubernetes client to discover the target API version and available fields.
Installation and Version Management
Installation is managed through Terraform 0.13+ provider mechanics. The provider can be installed and managed automatically by Terraform. A sample versions.tf file is referenced as a pattern for declarative provider pinning. If the one-liner approach is not desired, a binary can be downloaded for the system from the release page, then placed either at the root of the Terraform folder or in the Terraform plugin folder on the system.
The installation choice influences team workflow. Automatic management reduces manual binary distribution and ensures version consistency across workstations and CI. Manual binary placement gives operators explicit control over the exact provider build used in production pipelines.
See User Guide for details on installation and all the provided data and resource types. The User Guide is the canonical reference for installation procedures and the complete set of data and resource types.
Development Workflow and Build System
Development of the provider requires Go installed on the machine, version 1.12+ is required. The environment also requires correctly setup GOPATH, as well as adding $GOPATH/bin to $PATH. To compile the provider, run make build.
make build
The build command triggers the Build System defined in main.go, go.mod, and Makefile. The Makefile orchestrates compilation targets, tagging, and packaging for distribution.
The development workflow is important for contributors because the provider must compile against multiple Go versions and target platforms. The GOPATH requirement reflects the historical Go workspace layout used by the project.
The provider can be installed and managed automatically by Terraform. The one-liner installation approach is presented as the default path for consumers, while developers use make build for local iteration.
The provider can be downloaded from the release page, then either placed at the root of your Terraform folder or in the Terraform plugin folder on your system.
chmod +x terraform-provider-kubectl*
rm -rf terraform-provider-kubectl-tmp
rm -rf terraform-provider-kubectl.zip
The snippet reflects manual binary handling steps that are documented for users who prefer explicit control over provider binaries.
Testing, Quality Assurance, and CI/CD
The provider includes comprehensive testing, build automation, and quality assurance systems that ensure reliability across multiple Kubernetes and Terraform versions.
The Testing Framework contains unit and acceptance tests in kubernetes/provider_test.go and _examples/. Unit tests validate internal YAML parsing, templating, and client interactions. Acceptance tests exercise real Kubernetes API interactions in controlled environments.
The CI/CD Pipeline uses .github/workflows/ and scripts/ to automate testing and releases. Automated testing runs against the version matrix that covers K8s versions and Terraform versions. This matrix testing ensures that new releases do not break compatibility with supported platforms.
Code Quality is enforced through gofmt, go vet, and errcheck. These tools provide static analysis and formatting consistency across contributions.
The version matrix supports the last 7 Kubernetes releases and last 4 stable Terraform versions, with automated testing. The breadth of the matrix directly reduces upgrade risk for enterprises running mixed version clusters.
Provider Configuration Patterns
A provider block for kubectl commonly includes host, clustercacertificate, token, and loadconfigfile settings. The example shows:
provider "kubectl" {
host = var.eks_cluster_endpoint
cluster_ca_certificate = base64decode(var.eks_cluster_ca)
token = data.aws_eks_cluster_auth.main.token
load_config_file = false
}
Setting loadconfigfile to false prevents the provider from reading local kubeconfig files, which is essential in CI environments where authentication is supplied via variables.
The provider operates as a bridge between Terraform’s resource management lifecycle and Kubernetes’ API server. The bridge pattern means Terraform state tracks the desired YAML, while the provider translates state changes into Kubernetes API calls.
Dynamic Kubernetes clients discover API resources at runtime. This avoids the need for a static provider schema for every Kubernetes API group and allows custom resources to be managed immediately after they are installed in the cluster.
Rollout monitoring is part of lifecycle management. After creation or update, the provider can wait for rollout completion, which aligns Terraform’s apply phase with Kubernetes readiness.
Relationship to Hashicorp Kubernetes Provider
Reference material also discusses the Hashicorp Kubernetes provider, which is a distinct provider for interacting with resources supported by Kubernetes.
A tutorial configuration for the Hashicorp provider uses:
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
version = "~> 3.0"
}
}
}
Variables are defined for host, clientcertificate, clientkey, and clustercacertificate.
variable "host" {
type = string
}
variable "client_certificate" {
type = string
}
variable "client_key" {
type = string
}
variable "cluster_ca_certificate" {
type = string
}
Provider configuration decodes base64 values:
provider "kubernetes" {
host = var.host
client_certificate = base64decode(var.client_certificate)
client_key = base64decode(var.client_key)
cluster_ca_certificate = base64decode(var.cluster_ca_certificate)
}
To properly configure this provider, you need to define the variables. First, view your kind cluster information:
kubectl config view --minify --flatten --context=kind-terraform-learn
The output includes apiVersion, clusters with certificate-authority-data and server, contexts, current-context, kind Config, preferences, and users with client-certificate-data.
Kubernetes is an open-source workload scheduler with focus on containerized applications. You can use the Terraform Kubernetes provider to interact with resources supported by Kubernetes.
In the tutorial, you will learn how to interact with Kubernetes using Terraform, by scheduling and exposing a NGINX deployment on a Kubernetes cluster. You will also manage custom resources using Terraform.
While you could use kubectl or similar CLI-based tools to manage your Kubernetes resources, using Terraform has benefits:
- Unified Workflow - If you are already provisioning Kubernetes clusters with Terraform, use the same configuration language to deploy your applications into your cluster.
- Full Lifecycle Management - Terraform doesn't only create resources, it updates, and deletes tracked resources without requiring you to inspect the API to identify those resources.
- Graph of Relationships - Terraform understands dependency relationships between resources. For example, if a Persistent Volume Claim claims space from a particular Persistent Volume, Terraform won't attempt to create the claim if it fails to create the volume.
The tutorial assumes some basic familiarity with Kubernetes and kubectl. It also assumes that you are familiar with the usual Terraform plan/apply workflow.
The distinction between the two providers is important for architectural choices. The Hashicorp Kubernetes provider offers first-party resources with schema validation, while terraform-provider-kubectl offers YAML-first flexibility and drift detection for any manifest.
Operational Considerations
Installation steps for a tutorial environment include creating a working directory.
mkdir learn-terraform-deploy-nginx-kubernetes
cd learn-terraform-deploy-nginx-kubernetes
Then create a new file named kubernetes.tf and add the configuration. The cloud provider tabs will configure the Kubernetes provider using cloud-specific auth tokens.
The provider’s YAML processing at its core means that changes to manifests are diffed at the YAML level before API submission. This reduces unnecessary API calls and preserves Kubernetes field management semantics.
Multi-document parsing and templating enable teams to maintain manifests in version control and apply them declaratively via Terraform, aligning with GitOps practices.
Drift detection ensures that manual kubectl edits are surfaced during Terraform plan, which supports compliance and audit requirements.
Conclusion
The terraform-provider-kubectl project delivers a YAML-centric bridge between Terraform and Kubernetes. The kubectl_manifest resource provides free-form YAML processing with creation, update, deletion, and drift detection. YAML processing with templating, multi-document parsing, and manifest validation is centralized in the provider’s core. Dynamic Kubernetes clients enable discovery and lifecycle management including rollout monitoring. Build automation, testing frameworks, CI/CD pipelines, and code quality systems sustain reliability across the last 7 Kubernetes releases and last 4 stable Terraform versions. Installation can be automatic via Terraform 0.13+ or manual via release binaries. Development requires Go 1.12+ and make build. The provider has been used by many large Kubernetes installations to completely manage the lifecycle of Kubernetes resources. The architectural separation from the Hashicorp Kubernetes provider offers a complementary path for teams prioritizing native YAML fidelity over schema-driven resources.