The Pulumi engine is designed as a lean orchestrator, which means it does not natively contain the logic required to understand every cloud API or every programming language. Instead, it utilizes a sophisticated extensibility mechanism known as the plugin system. This architecture allows the Pulumi engine to communicate in a uniform manner with a diverse array of external components, including language runtimes, resource providers, and specialized tooling. By offloading the specific implementation details to plugins, Pulumi ensures that the core engine remains stable and lightweight while the ecosystem can grow rapidly as new cloud services and languages emerge. These plugins operate as separate processes, isolating the core engine from potential crashes or memory leaks within a specific provider. To facilitate this communication between the engine and the plugin processes, Pulumi relies on gRPC, a high-performance remote procedure call framework that ensures efficient, typed, and language-agnostic data exchange.
The Five Categories of Pulumi Plugins
Pulumi classifies its plugins into five distinct categories, each serving a specific purpose within the infrastructure-as-code lifecycle. This categorization ensures a clean separation of concerns, from the code you write to the actual API calls made to a cloud provider.
Resource Plugins
Resource plugins, frequently referred to as providers, are the most common type of plugin encountered by users. These plugins expose standardized interfaces that allow the Pulumi engine to perform Create, Read, Update, and Delete (CRUD) operations on cloud resources.
- Direct Fact: Resource plugins are distributed as Pulumi packages and are listed in the Pulumi Registry.
- Impact Layer: This allows users to define infrastructure for AWS, Azure, GCP, and hundreds of other services using a consistent syntax, regardless of the underlying cloud API's complexity.
- Contextual Layer: Because resource plugins are separate processes, a failure in the AWS provider plugin will not crash the main Pulumi CLI or the language host process.
Beyond the official registry, users have the flexibility to create their own components and distribute them as resource plugins. This is particularly useful for organizations with internal platform teams that want to provide a set of "golden path" resources to their developers. These custom components can be published to the Pulumi IDP for internal discoverability or shared via direct Git references for more restricted distribution.
Language Plugins
Language plugins, also known as language hosts, are the bridge between the Pulumi engine and the code written by the developer. They allow the engine to execute programs written in various languages without the engine needing to understand the specific syntax or runtime of those languages.
- Direct Fact: A language plugin consists of a language executor binary named
pulumi-language-<language-name>and a language SDK. - Impact Layer: This enables developers to use the full power of general-purpose languages like Python, Node.js, Go, or .NET, including loops, conditionals, and standard libraries, to define their infrastructure.
- Contextual Layer: The executor binary is distributed with the Pulumi CLI, while the SDK handles the actual observation of resource registrations.
It is important to note that while most supported language plugins are installed automatically with the CLI, the community can develop and maintain separate language plugins to extend Pulumi's reach to other programming environments.
Analyzer Plugins
Analyzer plugins are specialized tools that scan Pulumi programs to identify potential issues before they are deployed. These plugins are the foundational technology that powers Pulumi Policy as Code.
- Direct Fact: Analyzer plugins scan for compliance, security vulnerabilities, and adherence to best practices.
- Impact Layer: Organizations can enforce guardrails, such as preventing the creation of unencrypted S3 buckets or restricting the deployment of resources to specific geographic regions, automatically during the
pulumi previewphase. - Contextual Layer: Because these are integrated into the plugin system, they can be updated independently of the main engine to reflect new security threats or updated regulatory requirements.
Converter Plugins
Converter plugins are designed for migration and interoperability. They transform existing infrastructure-as-code definitions from other ecosystem tools into Pulumi programs.
- Direct Fact: Converters can process Terraform files, Kubernetes YAML, and AWS CloudFormation templates.
- Impact Layer: This significantly lowers the barrier to entry for teams migrating to Pulumi, as they do not have to rewrite their entire infrastructure from scratch.
- Contextual Layer: These plugins are triggered specifically by the
pulumi convertcommand and are installed automatically when that command is invoked.
Tool Plugins
Tool plugins provide the integration layer between Pulumi and external third-party tools.
- Direct Fact: They extend Pulumi's capabilities to work within existing development workflows and toolchains.
- Impact Layer: This allows Pulumi to integrate with IDEs, CI/CD dashboards, and internal developer portals, ensuring that infrastructure management is not a siloed activity.
- Contextual Layer: An example of this is the
@pulumi/backstage-plugin-pulumi, which integrates Pulumi's resource visibility directly into the Backstage developer portal.
Technical Architecture and Internal Orchestration
The plugin system is not merely a collection of binaries but a structured architecture that manages a full lifecycle from discovery to cleanup. The engine uses a specific set of internal structures to handle how plugins are identified and executed.
Core Components of the Plugin System
The following table outlines the primary architectural components used by the Pulumi engine to manage plugins:
| Component | Type | Purpose |
|---|---|---|
workspace.PluginDescriptor |
struct | Identifies plugins by kind, name, version, and download URL. |
workspace.PluginSource |
interface | Abstracts plugin download from various remote sources. |
plugin.Host |
interface | Acts as the central manager for loading and caching plugin instances. |
The Lifecycle of a Plugin Process
The movement from a resource declaration in code to a running process follows a rigorous flow:
- Discovery: The engine identifies which plugins are required based on the resources defined in the program.
- Downloading: If the plugin is not in the local cache, the
workspace.PluginSourcehandles the retrieval from GitHub, GitLab, or a custom server. - Installation: The binary is placed in the appropriate local directory.
- Loading: The engine spawns a separate process for the plugin.
- Communication: A gRPC connection is established between the engine and the plugin process.
- Cleanup: Once the operation is complete, the engine handles the closing and caching of the process.
Plugin Installation and Management
While the Pulumi CLI is designed to handle most plugin operations automatically, there are several scenarios where manual intervention is required or beneficial.
Automatic vs. Manual Installation
Typically, when a user runs pulumi preview or pulumi up, the CLI scans the program for required providers and installs any missing ones into the plugin cache. However, manual installation is critical in the following contexts:
- Air-gapped environments: Where the machine has no outbound internet access to reach the Pulumi Registry.
- CI/CD Optimization: Pre-loading plugins into a custom Docker image to eliminate download times during pipeline execution.
- Beta Testing: Installing specific pre-release versions of a plugin to test new features.
- Community Plugins: Installing providers that are not hosted on the official Pulumi Registry.
CLI Management Commands
Pulumi provides a suite of commands to manage the local plugin environment.
pulumi plugin ls: This command lists all currently installed plugins on the system.pulumi plugin install [kind] [name] [version]: This allows for the manual installation of a specific plugin.pulumi plugin rm [kind] [name]: This removes a specific plugin from the local cache.pulumi plugin rm --all: This clears all cached plugins from the system.
Detailed Usage of the Install Command
The pulumi plugin install command is highly configurable to support various deployment scenarios.
pulumi plugin install [kind name [version]] [flags]
The following flags modify the behavior of the installation process:
--checksum string: Used to verify the SHA256 checksum of the plugin archive to ensure integrity and security.--exact: Forces the engine to install an exact version match, overriding the default behavior where a version greater than or equal to the specified version might be accepted.-f, --file string: Allows the installation of a plugin from a local binary, folder, or tarball, bypassing the remote download process.--reinstall: Forces the installation of a plugin even if it already exists in the local cache.--server string: Specifies a custom URL to download plugins from, which is essential for private enterprise registries.
Plugin Storage and Implementation
The physical location of plugins on a disk depends on how they were installed and their specific type.
Storage Locations
Pulumi utilizes two primary directories to organize its binaries:
~/.pulumi/bin: This directory stores plugins that ship directly with the Pulumi CLI. This includes all officially supported language plugins and standard policy plugins.~/.pulumi/plugins: This directory serves as the cache for all plugins installed by the user, whether they were installed automatically during apulumi upcommand or manually via the CLI.
Implementation Approaches
There are two primary ways that a Pulumi plugin is deployed as an executable:
- Naming Convention Executables: These are binaries that follow the strict naming pattern
pulumi-<kind>-<name>. For example, the AWS resource provider is identified aspulumi-resource-aws. - Source-based Plugins: These plugins contain a
PulumiPlugin.yamlconfiguration file. In this model, the engine does not run a pre-compiled binary directly but instead uses the specified runtime (via a language plugin) to execute the plugin logic.
Integration Example: Pulumi Backstage Plugin
A practical application of the tool plugin category is the integration of Pulumi into the Backstage developer portal. This allows teams to view their infrastructure state directly within their internal software catalog.
Installation Process
To integrate Pulumi into a Backstage application created with npx @backstage/create-app, the following steps must be performed:
Add the plugin package to the Backstage app:
yarn add --cwd packages/app @pulumi/backstage-plugin-pulumiUpdate the
EntityPage.tsxfile inpackages/app/src/components/catalog.
First, the following imports are added:
typescript
import {
isPulumiAvailable,
EntityPulumiCard,
EntityPulumiMetdataCard,
PulumiComponent,
PulumiDashboardPage,
} from '@pulumi/backstage-plugin-pulumi';
Next, a constant is created to handle the conditional rendering of the Pulumi component:
typescript
const pulumiContent = (
<EntitySwitch>
<EntitySwitch.Case if={isPulumiAvailable}>
<PulumiComponent/>
</EntitySwitch.Case>
</EntitySwitch>
);
The overviewContent constant is then modified by adding a Grid item:
typescript
<EntitySwitch>
<EntitySwitch.Case if={isPulumiAvailable}>
<Grid item md={6}>
<EntityPulumiCard variant="gridItem"/>
</Grid>
</EntitySwitch.Case>
</EntitySwitch>
The serviceEntityPage constant is updated to include the route:
typescript
<EntityLayout.Route path="/pulumi" title="Pulumi" if={isPulumiAvailable}>
{pulumiContent}
</EntityLayout.Route>
Finally, to add the standalone dashboard, the App.tsx file is updated:
```typescript
import { PulumiDashboardPage } from '@pulumi/backstage-plugin-pulumi';
// In your routes
```
Conclusion: Analysis of the Plugin Architecture
The Pulumi plugin system represents a strategic decision to prioritize flexibility and scalability over a monolithic design. By utilizing a gRPC-based, multi-process architecture, Pulumi effectively decouples the core orchestration engine from the volatile nature of cloud provider APIs. This means that an update to the AWS API does not require a full re-release of the Pulumi CLI; only the pulumi-resource-aws plugin needs to be updated.
The impact of this architecture is most visible in the developer experience. The automatic installation of providers removes the friction of environment setup for new users, while the pulumi plugin install command provides the granular control required by enterprise DevOps engineers managing air-gapped or highly secure environments. Furthermore, the introduction of analyzer and converter plugins demonstrates that the plugin system is not just for resource provisioning, but for the entire lifecycle of infrastructure governance and migration.
From a technical perspective, the reliance on workspace.PluginDescriptor and plugin.Host ensures that the engine can dynamically load components without knowing their implementation details. This "pluggable" nature allows Pulumi to support multiple languages (via language plugins) and multiple clouds (via resource plugins) simultaneously, making it one of the most versatile infrastructure-as-code tools available in the current market.