Localized Infrastructure Orchestration via Pulumi Local SDKs and Providers

The conceptual framework of Infrastructure as Code (IaC) typically revolves around the management of remote, cloud-native resources residing in data centers managed by third-party providers. However, a critical requirement for sophisticated engineering teams is the ability to manage resources that do not exist in a public registry or are tethered to the local execution environment. This requirement is addressed through the mechanism of local SDKs and specific local providers within the Pulumi ecosystem. By leveraging local SDKs, developers can define infrastructure resources once and make them available across all supported Pulumi languages without the immediate necessity of publishing to the centralized Pulumi Registry. This capability is foundational for teams developing custom providers, utilizing parameterized providers such as those derived from Terraform, or managing ephemeral local environments.

The distinction between a standard Pulumi package and a local SDK is rooted in the generation process. While standard packages are checked into a provider repository and published to the Pulumi Registry for global consumption, a local SDK is generated directly on the user's machine in the language of the active program. The underlying source or executable for the provider is often retrieved from a remote source, such as a Git repository or a registry, but the SDK itself—the language-specific bindings that allow a developer to write TypeScript, Python, or Go code—is constructed locally. This process is driven by a package schema, which serves as the blueprint that the Pulumi CLI uses to synthesize the appropriate SDK.

The Architecture of Local SDKs

Local SDKs serve as a bridge between a resource provider's schema and the end-user's code. This architecture is particularly vital in several high-impact scenarios that dictate the flexibility of an infrastructure pipeline.

• Parameterized Providers: When utilizing providers like the terraform-provider, local SDKs allow for the injection of specific parameters that may not be static in a published package.
• Component Packages: When working with components as packages, local SDKs enable the iterative development of higher-level abstractions before they are promoted to a shared organizational registry.
• Development and Testing: During the active development phase of a new provider, the ability to generate an SDK locally allows for rapid prototyping and testing of resource logic without the overhead of repeated registry publishing.
• Private Organizational Resources: For enterprises that maintain strict security postures, local SDKs facilitate the management of organization-specific resources that must never be exposed to a public registry.

The operational impact of this architecture is significant. By shifting the SDK generation to the local machine, Pulumi removes the bottleneck of registry latency and versioning bureaucracy during the inner-loop of development. This ensures that the developer's environment is always synchronized with the latest version of the provider's logic, whether that logic resides in a local binary or a remote Git tag.

Managing Local SDKs via the Pulumi CLI

The primary mechanism for integrating a local SDK into a Pulumi project is the pulumi package add command. This command performs three critical operations simultaneously to ensure the project remains consistent and reproducible. First, it locally generates the SDK in the currently selected Pulumi language. Second, it modifies the Pulumi.yaml project configuration file to include the package definition. Third, it provides the developer with the necessary instructions to link the generated package into the project's code.

The command syntax is structured as follows:

pulumi package add <provider|schema|path> [provider-parameter...] [flags]

Depending on the source of the provider, the implementation varies:

• Local Path: Using pulumi package add ./my-provider allows the CLI to execute the provider binary directly. This is the standard approach for developers creating custom providers, as it extracts the package schema from the binary on the fly.
• Schema File: Using pulumi package add ./my/schema.json generates an SDK directly from a JSON schema file. This is an essential workflow for component resources or when integrating with pre-defined schema definitions that are not yet bundled into a binary.

The ripple effect of this process is felt during team collaboration. When a collaborator clones a repository containing local SDK definitions in the Pulumi.yaml, they do not need to manually hunt for binaries. Instead, they run the pulumi install command. This triggers the automatic generation of all defined local SDKs on their specific machine, ensuring that the entire team is utilizing the exact same provider version and schema.

Lifecycle Management and SDK Upgrades

Infrastructure requirements are rarely static, necessitating a robust mechanism for upgrading local SDKs. To update a local SDK, the developer must re-run the pulumi package add command using the updated source or a specific version tag.

Examples of upgrade paths include:

• Terraform Provider Update: pulumi package add terraform-provider hashicorp/random 3.7.1
• Git Repository Tag Update: pulumi package add example.com/org/repo.git/[email protected]
• Schema Regeneration: pulumi package add ./my/schema.json

The execution of these commands results in the regeneration of the SDK based on the latest schema and an automatic update to the Pulumi.yaml file. Following the regeneration, the user must execute pulumi install to refresh the dependencies. This step is critical because if the new SDK version introduces breaking changes to the resource API, the developer will need to update their imports and resource declarations in the application code.

To verify the current state of local SDKs, developers have two primary options. They can manually inspect the packages section of the Pulumi.yaml file, or they use the command pulumi package info <package-name> to retrieve detailed metadata about the installed package.

Collaborative Team Workflow for Local SDKs

For engineering teams, the integration of local SDKs into a version control system requires a disciplined approach to prevent environment drift. The recommended workflow is as follows:

1. A designated developer identifies the need for a provider update or a new local resource.
2. The developer executes pulumi package add with the updated source.
3. The developer commits the modified Pulumi.yaml file to the source control system (e.g., GitHub, GitLab).
4. Other team members pull the latest changes from the repository.
5. Collaborators run pulumi install to regenerate the local SDKs on their own machines.

This workflow ensures that the Pulumi.yaml acts as the single source of truth for the infrastructure's dependency graph, while the actual SDK binaries—which can be bulky and platform-specific—are not committed to the repository.

Specialized Local Implementations: LocalStack and pulumi-local

A distinct specialized implementation of local orchestration is found in the pulumi-local wrapper, which is specifically designed to facilitate the use of LocalStack. LocalStack provides a fully functional local cloud stack, allowing developers to simulate AWS environments without incurring costs or requiring internet access.

The pulumilocal utility is installed via the Python package manager:

pip install pulumi-local

It is mandatory that a LocalStack instance is active on the local machine before utilizing this tool. The pulumilocal command is designed to be a drop-in replacement for the standard pulumi command, mirroring its usage and syntax. For comprehensive command references, users are directed to the pulumi -h man pages.

An example deployment sequence for a LocalStack project is as follows:

export PULUMI_CONFIG_PASSPHRASE=lsdevtest
export PULUMI_BACKEND_URL=file://pwd/myproj
mkdir myproj
pulumilocal new typescript -y -s lsdev --cwd myproj
pulumilocal stack select -c lsdev --cwd myproj
pulumilocal up --cwd myproj

In this sequence, the --cwd switch specifies the project directory. If the user is already operating within the project directory, this switch is redundant.

The core functionality of the pulumilocal wrapper manifests during deployment commands such as pulumilocal up, pulumilocal destroy, pulumilocal preview, and pulumilocal cancel. When these are executed, the wrapper script first runs the pulumi config command to augment the project configuration with the necessary LocalStack AWS settings. Only after the configuration is augmented does it execute the original Pulumi command.

Users can further tune the behavior of this wrapper using specific environment variables:

• AWSENDPOINTURL: Specifies the hostname and port of the target LocalStack instance.
• LOCALSTACK_HOSTNAME: (Deprecated) Used to define the target host, defaulting to localhost.
• EDGE_PORT: (Deprecated) Used to define the target port, defaulting to 4566.
• PULUMI_CMD: Defines the name of the Pulumi executable on the system PATH, defaulting to pulumi.
• CONFIG_STRATEGY: Determines the strategy used for merging configurations.

It is important to note a critical limitation: pulumi-local currently does not support the aws-native package.

The Local Provider and Migration Path

Apart from SDK generation and LocalStack wrappers, Pulumi offers a specific Local provider designed for managing local resources, such as files on a disk. This provider allows cloud programs to interact with the local filesystem as if it were a managed cloud resource.

The Local provider must be installed as a local package using the following command:

pulumi package add terraform-provider hashicorp/local

A significant architectural warning exists regarding the Local provider. Pulumi is fundamentally designed for remote resources that persist beyond the lifecycle of a single Pulumi run. Local resources can violate these assumptions. Depending on local state can make it exceedingly difficult to apply the same configuration across multiple different machines, as the local resources may not be universally available.

Furthermore, the original pulumi-local provider is deprecated as of version v0.1.6. Pulumi no longer maintains this version. For new users, the use of this provider is explicitly discouraged. Instead, users are directed toward the pulumi-command provider, which offers a more flexible and powerful method for executing local commands.

For those who must use the Local provider, the current recommended path is to generate it from the Local Terraform provider using the following command:

pulumi package add terraform-provider registry.opentofu.org/hashicorp/local <version>

The equivalent upstream version for [email protected] is search.opentofu.org/provider/hashicorp/local v2.5.2. For users migrating existing resources from the deprecated provider to the new version, the recommended procedure is to run pulumi import within a fresh stack that utilizes the locally built provider package.

Technical Summary of Local Package Installation Methods

The process of integrating local functionality into Pulumi can be categorized by the intent of the developer.

• For Custom Provider Development:
  The developer creates a provider binary and uses pulumi package add ./path-to-binary. This ensures the SDK is derived from the current state of the local binary.

• For Component-Based Architectures:
  The developer defines a JSON schema and uses pulumi package add ./schema.json. This allows the resource to be defined without a full provider binary.

• For LocalStack AWS Mocking:
  The developer installs the Python wrapper pulumi-local, which intercepts standard Pulumi commands to inject local AWS endpoint configurations.

• For Local File Management:
  The developer adds the hashicorp/local Terraform provider as a local package to enable filesystem interactions within their IaC code.

Analysis of Local State Risks and Mitigations

The use of local providers introduces a fundamental tension between the "Write Once, Run Anywhere" philosophy of IaC and the "Run Here" reality of local resources. When a Pulumi program depends on a local file or a local process, the state of that resource is tied to the specific machine executing the code.

If a project is moved from a developer's MacBook to a Linux-based CI/CD runner, any resource managed by the Local provider may fail if the file paths or environment permissions differ. This creates a "fragile state" where the Pulumi.stack.json describes a resource that does not exist in the new environment.

To mitigate these risks, the following strategies are recommended:

1. Use Local providers only for bootstrapping tasks, such as creating a temporary configuration file required for a remote provider.
2. Ensure that any local paths used in the code are parameterized via pulumi config, allowing different paths for different environments.
3. Prioritize the pulumi-command provider over the deprecated Local provider for executing shell scripts or local binaries, as it provides better control over execution context and error handling.
4. When using pulumi-local with LocalStack, utilize the PULUMI_BACKEND_URL environment variable to store the state in a local file (e.g., file://pwd/myproj) rather than a remote backend, ensuring the state and the simulated cloud remain co-located.

Conclusion

The capability to generate and utilize local SDKs transforms Pulumi from a cloud-only orchestrator into a versatile tool capable of managing the entire spectrum of technical resources. By decoupling the SDK generation from the registry publishing process, Pulumi empowers developers to iterate rapidly on custom providers and integrate private organizational resources without compromising security or speed. The introduction of specialized tools like the pulumi-local wrapper further extends this utility, providing a seamless bridge to LocalStack for cost-effective AWS development.

However, the transition from the deprecated pulumi-local provider to the Terraform-backed hashicorp/local or the more flexible pulumi-command provider highlights a critical evolution in the ecosystem: the move toward more robust, schema-driven local resource management. Developers must remain vigilant regarding the inherent risks of local state dependency, ensuring that their infrastructure remains portable and reproducible. By adhering to the disciplined workflow of updating Pulumi.yaml and executing pulumi install across teams, organizations can successfully leverage local SDKs to accelerate their development cycles while maintaining a rigorous standard of infrastructure integrity.

Sources

1. Pulumi Local SDKs Guide
2. LocalStack Pulumi-Local GitHub
3. Pulumi-Local GitHub Repository
4. Pulumi Registry Local Package