The orchestration of modern cloud infrastructure necessitates a robust mechanism for handling variables that fluctuate across different environments. Pulumi addresses this requirement through a sophisticated configuration system that moves beyond the limitations of traditional environment variables or static AWS Config setups. By decoupling the project-level definitions from stack-specific values, Pulumi allows engineers to maintain a single codebase while deploying tailored infrastructure to development, staging, and production environments. This architecture ensures that the underlying infrastructure as code (IaC) remains DRY (Don't Repeat Yourself) while providing the granularity needed for complex microservices deployments.
Project Configuration and the Pulumi.yaml Framework
At the foundational level of any Pulumi deployment is the project configuration. This is centrally managed within a file named Pulumi.yaml, located at the root of the project directory. This file serves as the primary manifest for the project, ensuring that the Pulumi CLI understands the basic parameters of the application before any resources are instantiated.
The Pulumi.yaml file contains several critical pieces of information:
- Project Name: This identifies the project across the Pulumi ecosystem, allowing for the organization of multiple projects within a single organization or account.
- Description: A human-readable summary of the project's purpose, which assists team members in understanding the intent of the infrastructure.
- Runtime: This specifies the execution engine required to run the code, such as Node.js or Python.
- Language Information: This defines the specific programming language used, ensuring the correct compiler or interpreter is invoked.
The impact of this centralized project configuration is the elimination of ambiguity during the deployment process. Because the runtime and language are explicitly defined, any developer cloning the repository can be confident that they are using the correct environment, thereby preventing "it works on my machine" failures. This file connects the code to the Pulumi engine, acting as the entry point for all subsequent configuration layers.
Stack Configuration and Environment Management
While the project configuration is global, stack configuration is specific to individual deployments. A stack is an isolated instance of a Pulumi project, typically representing an environment like dev, test, or prod. Pulumi manages these differences through stack-specific configuration files, which are automatically named using the pattern Pulumi.<stack-name>.yaml.
These files store key-value pairs that drive the behavior of the program. For example, a developer may require a small t3.micro instance for a development stack to minimize costs, but a larger m5.large instance for a production stack to ensure performance. Rather than hard-coding these values, the developer stores them in the respective .yaml files.
The operational consequences of this approach are significant:
- Version Control Integration: Because
Pulumi.<stack-name>.yamlfiles are intended to be committed to version control, the configuration becomes part of the audit trail. Changes to instance sizes or cluster counts are tracked via Git, allowing for easy rollbacks and peer reviews. - Environment Isolation: Each stack operates in its own silo. The configuration for the
prodstack is physically and logically separated from thedevstack, reducing the risk of accidental modifications to production environments. - Dynamic Behavior: By utilizing stack configurations, the same code can be deployed across multiple regions or clouds simply by switching the active stack and its associated configuration file.
Command Line Interface Configuration Management
The Pulumi CLI provides a comprehensive set of tools for managing these configuration values without requiring the user to manually edit YAML files, which reduces the risk of syntax errors.
The following table details the primary CLI commands used for configuration management:
| Command | Action | Description |
|---|---|---|
pulumi config set <key> <value> |
Addition/Update | Adds a new configuration value or updates an existing one for the active stack. |
pulumi config get <key-name> |
Retrieval | Retrieves the specific value associated with a configuration key. |
pulumi config rm <key> |
Removal | Deletes a configuration value from the current stack settings. |
pulumi config |
Listing | Lists all configuration values currently set for the specific stack. |
Beyond basic CRUD operations, the CLI offers advanced flags to modify how configuration is handled:
--config-file string: This allows the user to specify a custom configuration file instead of relying on the default Pulumi detection logic. This is particularly useful for testing different configuration sets without renaming files.--json: This flag emits the output in JSON format, which is essential for integrating Pulumi configuration into larger automation pipelines or external scripts.--show-secrets: By default, the CLI blinds secret values for security. This flag allows an authorized user to display the actual plaintext values of secrets during listing.-s, --stack string: This allows the user to operate on a specific stack regardless of which stack is currently selected as the active one.-C, --cwd string: This command allows Pulumi to run as if it had been started in a different directory, enabling centralized management of multiple project folders.
The use of these CLI tools creates a streamlined workflow where developers can rapidly iterate on environment variables. For instance, if a database username needs to be changed, running pulumi config set dbUsername new_user immediately updates the state, and the subsequent pulumi up command applies this change to the cloud provider.
The Programming Model for Configuration Retrieval
Once configuration values are set via the CLI or YAML files, they must be accessed within the code to provision resources. Pulumi provides a Config object to bridge the gap between the static YAML files and the execution logic.
In a TypeScript environment, the configuration is accessed as follows:
typescript
import * as pulumi from '@pulumi/pulumi';
const config = new pulumi.Config();
export const dbUsername = config.require('dbUsername');
export const dbPassword = config.require('dbPassword');
The config.require method is a critical component of this model. It retrieves a required configuration value. If the value is missing from the stack configuration, Pulumi will throw an error and halt the deployment. This prevents the infrastructure from being deployed in an incomplete or broken state, as the user is forced to run pulumi config set dbUsername <value> before the deployment can proceed.
In addition to the Config object, Pulumi programs can access standard shell environment variables. In Node.js, these are available via process.env, and in Python, they are available via os.environ. This allows for dynamic behavior based on the environment in which the Pulumi CLI is executing, providing an additional layer of flexibility for CI/CD pipelines.
Secrets Management and Encryption
Handling sensitive data such as API keys, passwords, and certificates requires a higher level of security than standard configuration. Pulumi integrates secrets management directly into its configuration system to ensure that sensitive data is never stored in plaintext within version control.
Internal Secrets Handling
When a value is marked as a secret, Pulumi encrypts it. When listing configuration via the CLI, these values appear as blinded (encrypted) strings. To decrypt these values, the Pulumi engine uses a secrets provider.
Users have several options for encryption:
- Passphrase Encryption: Pulumi can encrypt secrets using a passphrase. This is a simple method for small teams or local development.
- Custom Secrets Providers: For enterprise-grade security, Pulumi allows the use of external providers. When creating a new stack, the
--secrets-providerargument can be used to define the encryption mechanism, such as using AWS KMS.
External Secrets Integration with HashiCorp Vault
For organizations requiring centralized secrets management, Pulumi supports integration with tools like HashiCorp Vault. This allows the infrastructure code to retrieve secrets dynamically from a secure vault rather than storing them in Pulumi's own configuration files.
The implementation of a Vault-based secret retrieval is as follows:
typescript
import * as pulumi from '@pulumi/pulumi';
import * as vault from '@pulumi/vault';
const vaultServer = new vault.Server({
address: 'https://vault.example.com',
tlsCert: 'path/to/tls/cert',
});
const secret = vaultServer.getSecret({
path: 'db/credentials',
});
export const dbUsername = secret.data.dbUsername;
export const dbPassword = secret.data.dbPassword;
In this workflow, Pulumi does not store the secret; it stores the path to the secret within the Vault server. This reduces the attack surface, as the secrets reside in a dedicated, hardened security tool. The impact is a significant increase in security posture, as rotation and access control are managed by the Vault server rather than the IaC tool.
Specialized Pulumi CLI Configuration Keys
The Pulumi CLI recognizes several built-in configuration keys that modify the behavior of the engine itself. These are not user-defined variables but rather system-level instructions.
Default Provider Control
Pulumi normally attempts to automatically manage providers for the cloud services being used. However, there are scenarios where this default behavior is undesirable.
The pulumi:disable-default-providers key allows users to specify which providers should be disabled. This is configured in the YAML file as follows:
yaml
pulumi:disable-default-providers:
- aws
- kubernetes
By disabling default providers, developers gain absolute control over the provider instance being used, which is essential when dealing with complex multi-account setups or specific region overrides.
Stack Tagging
To improve the organization and searchability of resources in the cloud, Pulumi supports stack tags. These tags are read by the CLI and automatically applied to the stack during pulumi up or pulumi refresh operations.
Example configuration for tags:
yaml
pulumi:tags:
company: "Some LLC"
team: Ops
The impact of this is the ability to track costs and resource ownership at a glance within the cloud provider's console. It is important to note that the Pulumi CLI only creates or updates tags listed in the config. If a tag is removed from the stack configuration, it must be manually removed from the stack in Pulumi Cloud.
Pulumi ESC and Advanced Configuration Inheritance
As projects scale to dozens of microservices, configuration sprawl becomes a significant challenge. Duplicate configuration across multiple stacks leads to maintenance overhead and the risk of inconsistency. Pulumi ESC (Environments, Secrets, and Configuration) provides a solution for this by allowing common configuration and secrets to be shared across various stack configuration files.
With Pulumi ESC, a stack can import an environment:
```yaml
environment:
- test
config:
normal pulumi config
```
The precedence logic for this system is strictly defined: if a configuration key is set both in an imported environment (via ESC) and explicitly in the local stack configuration, the explicit stack value takes precedence. This allows for a "global default" with "local overrides," mirroring the structure of .NET appsettings while maintaining the flexibility of dynamic parsing.
Conclusion
The Pulumi configuration system is a multi-layered architecture designed to handle the complexity of modern cloud deployments. By splitting settings into project-level Pulumi.yaml and stack-level Pulumi.<stack-name>.yaml files, Pulumi ensures that the codebase remains portable while the environment remains specific. The integration of a powerful CLI for management, a strongly-typed programming model for retrieval, and a robust secrets framework—ranging from simple passphrases to enterprise HashiCorp Vault integrations—allows organizations to scale their infrastructure without sacrificing security or consistency. The addition of Pulumi ESC further enhances this by eliminating duplication through environment inheritance, ensuring that configuration management remains a catalyst for deployment speed rather than a bottleneck of manual updates.