The conceptualization of infrastructure as code has shifted from static configuration files to the utilization of general-purpose programming languages. Central to this shift is the mechanism of the Pulumi package. A Pulumi package is not a monolithic entity but a sophisticated dual-layered system designed to bridge the gap between high-level language abstractions and the low-level API calls required by cloud service providers. By decoupling the provider logic from the language-specific interface, Pulumi allows developers to maintain strong typing and IDE support while managing a vast array of resources across disparate cloud environments. This architecture ensures that as cloud providers update their APIs, the underlying provider plugins can be updated without necessarily requiring a total rewrite of the consuming application's logic, provided the SDK remains compatible.
The Dual-Component Architecture of Pulumi Packages
A Pulumi package is fundamentally composed of two distinct parts. This separation is critical because it allows a single provider to be consumed by any language supported by the Pulumi ecosystem, regardless of the language used to write the provider itself.
The first component is the provider plugin. This is the engine of the package. It contains the actual Pulumi code responsible for interacting with the target API. The provider plugin is written in a language supported by Pulumi and is responsible for three primary elements:
- Custom Resources: These are the core building blocks of infrastructure. The provider plugin defines the Create, Read, Update, and Delete (CRUD) operations for these resources. For example, when a user defines an AWS S3 bucket in their code, the provider plugin executes the specific API calls to Amazon Web Services to ensure that bucket exists and matches the desired state.
- Functions: Unlike resources, which represent physical or virtual assets, functions are used to query cloud providers for data. This allows a program to fetch existing information about an environment—such as the current VPC ID or an available AMI—and use that data as an input for other resources.
- Components: These are higher-level abstractions. Components encapsulate one or more custom resources (or other components) into a reusable unit. This allows organizations to create "golden paths" or standardized infrastructure patterns, such as a "SecureWebBucket" component that automatically bundles an S3 bucket with specific encryption and public access block settings.
The second component is the SDK. The SDK serves as the language-specific interface that the developer interacts with in their IDE. It is generated directly from the provider's schema file. This schema acts as the "source of truth," defining every available resource, property, and method. Because the SDK is generated from this schema, it can provide strong typing, auto-completion, and compile-time validation.
Depending on the package, the SDK may be handled in one of two ways:
- Published SDKs: These are pre-generated and hosted on standard language package feeds. For instance, Python users find them on PyPI, and Node.js users find them on npm. This provides a seamless installation experience similar to any other third-party library.
- Local SDKs: In cases where a published SDK does not exist—such as for custom internal components or specific Terraform providers—the Pulumi CLI can generate the SDK locally on the developer's machine by combining the local environment with the package schema.
The Pulumi Registry and Consumption Models
The Pulumi Registry serves as the centralized directory for all available Pulumi packages. It acts as the primary discovery mechanism where developers can find resources for major cloud providers, SaaS platforms, and community-contributed components. Each package listing in the Registry includes a dedicated Installation & Configuration page, providing the exact commands needed to integrate the provider into a project.
There are two primary workflows for adding these packages to a program, depending on how the SDK is distributed.
Using Packages with Published SDKs
The vast majority of packages in the Pulumi Registry, especially those for major cloud providers like AWS, Azure, and Google Cloud, have published SDKs. To use these, the developer employs their language's native package manager. This integrates Pulumi resource management directly into the existing dependency lifecycle of the project.
- Node.js (JavaScript/TypeScript): Developers use
npm install @pulumi/awsoryarn add @pulumi/aws. - Python: Developers utilize
pip install pulumi_aws. - Go: Developers run
go get github.com/pulumi/pulumi-aws/sdk/v7. - .NET: Developers execute
dotnet add package Pulumi.Aws.
Using Local SDKs
For packages that lack a published SDK—such as specific Terraform providers or internal company components—Pulumi utilizes a local generation mechanism. This is triggered by the command pulumi package add.
When pulumi package add is executed, several automated steps occur:
1. The Pulumi CLI downloads the necessary provider plugin.
2. The CLI reads the package schema.
3. The CLI generates a language-specific SDK locally within the project environment.
4. The CLI adds a formal reference to the package within the Pulumi.yaml file.
This ensures that the project remains portable. When another developer clones the repository, they do not need to manually hunt for the local SDK; they simply run the installation command to regenerate the environment.
Specialized AWS Resource Management
The pulumi-aws package is a primary example of a high-scale provider plugin. It provides a strongly-typed interface to the entirety of the Amazon Web Services ecosystem. This means that almost every property and resource available in the AWS API is exposed through the Pulumi SDK.
The scope of resources managed by the pulumi-aws package includes, but is not limited to, the following:
- apigateway: For creating and managing API endpoints.
- cloudformation: For interacting with AWS CloudFormation stacks.
- EC2: For managing virtual servers and networking.
- ECS: For orchestrating containerized applications.
- iam: For defining identities, users, and access policies.
- lambda: For deploying serverless functions.
Beyond simple resource mapping, the pulumi-aws package includes convenience APIs designed to reduce developer friction and prevent common configuration errors. A notable example is the aws.lambda.CallbackFunction class. This class allows developers using JavaScript or TypeScript to create an AWS Lambda function directly from a function object that possesses the correct signature, removing the need for separate build and upload steps for simple function logic.
Dependency Lifecycle and Installation
Managing the installation of Pulumi packages requires an understanding of how standard package managers interact with Pulumi's internal tracking.
The pulumi install command
The pulumi install command is the recommended method for finalizing a project's environment. It serves as a unified installation wrapper. Rather than requiring a developer to run both a language-specific manager (like pip or npm) and a Pulumi-specific command, pulumi install handles both.
Specifically, pulumi install performs the following:
- It processes standard package manager dependencies listed in files like package.json or requirements.txt.
- It identifies any local SDKs defined in the Pulumi.yaml file.
- It ensures that all necessary provider plugins are downloaded and available to the Pulumi engine.
This command is critical in several scenarios:
- Project Initialization: When a project is first cloned from source control, pulumi install ensures the local environment matches the versioning defined in the configuration.
- Dependency Updates: After modifying package versions, this command synchronizes the actual installed files with the requested versions.
- Team Synchronization: By running this command, all contributors to a project are guaranteed to be using the same provider versions, eliminating "works on my machine" bugs caused by version drift.
Versioning and Upgrade Strategies
Upgrading Pulumi packages requires different approaches depending on whether the SDK is published or local. Failure to upgrade correctly can lead to state mismatches or broken infrastructure deployments.
Upgrading Published SDKs
For packages available on public feeds, the process mirrors standard library updates. Developers use the following commands:
- Node.js:
npm install @pulumi/aws@latest - Python:
pip install --upgrade pulumi-aws - Go:
go get github.com/pulumi/pulumi-aws/sdk/v7@latest(Note: Go requires the major version, such as v7, to be specified in the path). - .NET:
dotnet add package Pulumi.Aws
After performing these updates via the package manager, it is a best practice to run pulumi install to ensure the underlying provider plugins are aligned with the new SDK versions.
Upgrading Local SDKs
For local SDKs, the upgrade process involves re-registering the package. The pulumi package add command is used again, but with a specific version constraint.
Examples of local upgrades:
- For a Terraform provider: pulumi package add terraform-provider hashicorp/random 3.7.1
- For a component from a git repository: pulumi package add example.com/org/repo.git/[email protected]
Executing these commands triggers the regeneration of the SDK and automatically updates the version entry in the Pulumi.yaml file. To verify which versions of local SDKs are currently active in a project, a developer should inspect the packages section of the Pulumi.yaml file.
Runtime Requirements for Different Languages
While Pulumi abstracts much of the underlying complexity, the language chosen for the program dictates the runtime requirements for the generated or published SDKs.
The following table outlines the runtime dependencies associated with various Pulumi language packages:
| Language/Package Type | Runtime Requirement | Detail |
|---|---|---|
| Go | Go Language Runtime | Required for both published and local SDKs to execute. |
| .NET (Runtime-included) | None | Recommended approach; binaries include necessary runtimes. |
| .NET (Runtime-dependent) | .NET Runtime | Required if binaries are not compiled as runtime-included. |
| Java | JVM | A Java Virtual Machine is mandatory for all Java packages. |
| YAML | None | No specific runtime requirements for YAML-based configurations. |
Authoring and Distributing Packages
Pulumi allows developers to transition from being consumers of packages to being authors. There are two primary paths for authoring, depending on the intended audience and the goal of the package.
Authoring a Pulumi Component
A component is a higher-level abstraction used to group resources. For example, a company might create a "StandardNetwork" component that includes a VPC, subnets, and routing tables. When authoring a component for distribution:
- Internal Sharing: If the component is for a small team, local SDKs may suffice.
- Broad Distribution: If the component is intended for the wider organization or the public community, the author should consider publishing the SDK to a package feed or the Pulumi Registry.
Authoring a Pulumi Provider
A provider is a more complex undertaking. This involves creating a package that allows consumers to manage resources for an entirely different cloud or SaaS provider. The author must define the provider plugin, implement the CRUD operations for the resources, and provide the schema file from which SDKs are generated. Once the provider is complete, the author can optionally publish it to the Pulumi Registry for public consumption.
Internal Infrastructure and SDK Utilities
Within the Pulumi core ecosystem, there are specialized packages and internal utilities that support the broader platform. These are typically not intended for end-user consumption but are vital for the operation of the CLI and engine.
The pulumi-internal package and its sub-packages contain helper utilities reserved for use exclusively within Pulumi's own codebase. One such sub-package is pulumi-internal/gsync, which provides generic synchronization capabilities.
Other critical internal structures include:
- pulumi/provider: This package contains the essential logic for establishing Remote Procedure Call (RPC) connections to provider plugins. This is the communication bridge that allows the Pulumi engine to tell a plugin to create a resource.
- tools/automation: This contains the "Automation API," which provides auto-generated wrappers around the Pulumi CLI. This allows developers to embed Pulumi deployment logic directly into their own application code rather than running CLI commands manually.
- tools/automation/boilerplate/testing: A dedicated package providing the necessary scaffolding for testing automation logic.
- Language-specific cores: The main repository contains core logic for nodejs, proto, and python to ensure consistent behavior across the SDKs.
Summary of Package Management Workflow
The complete lifecycle of a Pulumi package, from discovery to upgrade, follows a rigorous path to ensure infrastructure stability.
The workflow typically begins at the Pulumi Registry. Once a provider is selected, the developer installs the SDK via a language manager (e.g., npm, pip) or via pulumi package add for local/Terraform providers. The environment is then locked in using pulumi install, which synchronizes the Pulumi.yaml file with the actual installed plugins.
When the infrastructure evolves, the developer upgrades the SDK through the same channels—using @latest for published packages or version-specific tags for local ones. The process concludes with a final pulumi install to verify that the provider plugins and the SDKs are in sync, ensuring that the next pulumi up command executes against the correct API versions.