Architectural Decoupling via Pulumi StackReference

The modern infrastructure landscape demands a shift from monolithic state files toward a modular, decoupled architecture. As organizations scale, the blast radius of a single infrastructure project becomes an unacceptable risk, and the necessity for independent team autonomy grows. Pulumi provides a sophisticated mechanism to solve this tension through the implementation of Stack References. A StackReference is a programmatic bridge that allows one Pulumi project to access the outputs of another project at runtime. This functionality effectively decouples the lifecycle of different infrastructure components—such as separating the foundational networking layer from the application deployment layer—while still maintaining a strict data dependency between them.

In traditional Infrastructure as Code (IaC) paradigms, such as Terraform, this pattern is analogous to using a Remote State datasource. By allowing a project to ingest the outputs of a separate state file, Pulumi ensures that changes to a high-level application service do not require a re-evaluation or a potential risk to the underlying VPC or Database cluster. This architectural pattern is essential for large-scale enterprises where different teams manage different layers of the stack, allowing a networking team to iterate on firewall rules independently of a development team iterating on containerized microservices.

The Fundamental Mechanics of StackReference

At its core, a StackReference is a class used to retrieve the exported outputs of a specific Pulumi stack. When a Pulumi program exports a value—such as a URL, an ID, or a connection string—that value is stored in the stack's state. A second Pulumi program can then instantiate a StackReference object, providing the unique identifier of the target stack, and request those specific outputs.

This process relies on a backend storage system. The Pulumi CLI must be authenticated to a backend where the state of both the source stack (the one providing the data) and the destination stack (the one consuming the data) resides. This lookup occurs during the execution of the Pulumi program, meaning the source stack must already exist and have been deployed before the consuming stack can successfully retrieve its outputs.

The relationship between the two stacks is unidirectional. The consuming stack depends on the source stack, but the source stack remains entirely unaware of which other projects are referencing its outputs. This creates a clean dependency graph that prevents circular dependencies and simplifies the deletion or modification of infrastructure.

Backend Infrastructure and State Management

The ability of a StackReference to locate and retrieve data is entirely dependent on the configured backend. The backend serves as the source of truth for the state of all managed resources.

The Managed Pulumi Cloud

The default and most common backend is the managed Pulumi Cloud (app.pulumi.com). This service provides a sophisticated user interface, deployment histories, and deep integration into CI/CD pipelines. When using the managed cloud, the StackReference uses a fully qualified name to locate the target stack. This identifier typically follows the pattern organization/project/stack.

For example, in a setup where a user named danielwarddev has a project called PulumiCSharp and a stack named dev, the fully qualified name would be danielwarddev/PulumiCSharp/dev. The Pulumi CLI verifies the identity and permissions of the user via the credentials file located at %USERPROFILE%/.pulumi/credentials.json on Windows systems to ensure the requester has the necessary rights to access the state.

Custom Backends and Self-Hosted Options

For organizations with strict regulatory requirements or a preference for self-management, Pulumi supports various custom backends:

• Azure Blob Storage: State files are stored as blobs within an Azure storage account.
• Amazon S3: State files are stored as objects within an S3 bucket.
• Google Cloud Storage: State files are stored in GCS buckets.
• Local Files: State is stored on the local filesystem (primarily used for testing).
• Self-hosted Pulumi Cloud: A private instance of the management plane.

When utilizing a custom backend like Azure Blob Storage, the behavior of StackReferences changes. In the managed Pulumi Cloud, the organization name is a core part of the naming convention. However, when using a custom backend, the CLI does not allow slashes for stack names in the same way. To reference a stack in a custom backend, a different syntax is employed. In C#, for instance, a developer might use a StackReferenceArgs object:

csharp var stackReference = new StackReference("shared", new StackReferenceArgs { Name = "demo.dev" });

It is critical to note a significant limitation regarding custom backends: currently, there is no supported method to reference a stack that resides in a completely separate blob storage container. The referencing stacks must share the same backend environment to discover one another.

Implementation Patterns Across Languages

Pulumi's multi-language support allows StackReferences to be implemented in TypeScript, Python, Go, C#, Java, and more. Regardless of the language, the logic remains consistent: instantiate the reference, call the output method, and use the resulting value.

TypeScript Implementation

In TypeScript, the process involves importing the Pulumi SDK and using the pulumi.Config class to avoid hardcoding stack names. This is vital for maintaining environment parity (e.g., dev, staging, prod).

```typescript
import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
const stack = pulumi.getStack();
const org = config.require("org");

const stackRef = new pulumi.StackReference(${org}/my-first-app/${stack});
export const shopUrl = stackRef.getOutput("url");
```

In this snippet, the org variable is pulled from the configuration file, allowing the same code to work across different organizational units. The getOutput method retrieves the specific value exported by the my-first-app project.

C# Implementation

C# developers can utilize the RequireOutput method to ensure that the value is not null and to cast it to the appropriate type, typically a string.

csharp var config = new Config(); var stackName = config.Require("stackName"); var stackRef = new StackReference(stackName); var outputValue = stackRef.RequireOutput("outputName").ToString();

The use of RequireOutput acts as a validation step, causing the deployment to fail immediately if the expected output is missing from the source stack, rather than failing later with a null reference exception during resource creation.

Advanced Use Cases and Architectural Scenarios

StackReferences enable complex infrastructure topologies that would be cumbersome or impossible with a single monolithic project.

Cross-Region Global Deployments

One of the most powerful applications of StackReferences is managing resources across multiple geographic regions. A global application may require a cluster in both the US-West and US-East regions. Instead of duplicating the entire cluster logic, a developer can reference regional cluster stacks from a global application project.

```typescript
// applications/global-app/index.ts
const usWestCluster = new pulumi.StackReference(
"organization/cluster/us-west-2-prod"
);
const usEastCluster = new pulumi.StackReference(
"organization/cluster/us-east-1-prod"
);

const usWestProvider = new k8s.Provider("us-west", {
kubeconfig: usWestCluster.getOutput("kubeconfig")
});

const usEastProvider = new k8s.Provider("us-east", {
kubeconfig: usEastCluster.getOutput("kubeconfig")
});

function deployApp(name: string, provider: k8s.Provider) {
return new k8s.apps.v1.Deployment(name, {
spec: {
replicas: 3,
selector: { matchLabels: { app: "myapp" } },
template: {
metadata: { labels: { app: "myapp" } },
spec: {
containers: [{
name: "app",
image: "myapp:latest"
}]
}
}
}, { provider });
}

const usWestDeployment = deployApp("app-us-west", usWestProvider);
const usEastDeployment = deployApp("app-us-east", usEastProvider);
```

In this scenario, the global application project does not manage the Kubernetes clusters themselves. Instead, it references the kubeconfig output from the regional clusters to instantiate K8s Providers. This allows the cluster infrastructure to be upgraded or rotated without affecting the application deployment logic.

Tiered Infrastructure: Networking, Cluster, and App

A common enterprise pattern is the three-tier split: Networking, Cluster, and Application. This ensures that a mistake in the application deployment cannot accidentally delete the VPC or the underlying EKS/GKE cluster.

1. Networking Stack: Deploys the VPC, Subnets, and NAT Gateways.
2. Cluster Stack: References the Networking stack to place the cluster in the correct subnets.
3. Application Stack: References the Cluster stack to deploy containers to the cluster.

Example of a Cluster stack consuming Networking outputs:

```typescript
// infrastructure/cluster/index.ts
import * as pulumi from "@pulumi/pulumi";
import * as eks from "@pulumi/eks";

const config = new pulumi.Config();
const networkingStackName = config.require("networkingStack");

const networkingStack = new pulumi.StackReference(networkingStackName);

const vpcId = networkingStack.getOutput("vpcId");
const privateSubnetIds = networkingStack.getOutput("privateSubnetIds");
const publicSubnetIds = networkingStack.getOutput("publicSubnetIds");

const cluster = new eks.Cluster("main", {
vpcId: vpcId,
privateSubnetIds: privateSubnetIds,
publicSubnetIds: publicSubnetIds,
instanceType: "t3.medium",
desiredCapacity: 3,
minSize: 1,
maxSize: 5,
createOidcProvider: true
});

export const clusterName = cluster.eksCluster.name;
export const kubeconfig = cluster.kubeconfigJson;
```

To make this work, the user must configure the reference in the CLI:

bash cd infrastructure/cluster pulumi stack init prod pulumi config set networkingStack organization/networking-stack/prod pulumi up

Application-to-Backend Integration

In a microservices environment, a frontend application needs the URL of a backend API. Rather than hardcoding this URL or using a separate configuration management tool, the frontend project can reference the backend project.

```typescript
const backendStack = new pulumi.StackReference(
config.require("backendStack")
);

const backendServiceUrl = backendStack.getOutput("serviceUrl");

const deployment = new k8s.apps.v1.Deployment("frontend", {
spec: {
selector: { matchLabels: { app: "frontend" } },
template: {
metadata: { labels: { app: "frontend" } },
spec: {
containers: [{
name: "frontend",
image: "frontend:latest",
env: [{
name: "API_URL",
value: backendServiceUrl
}]
}]
}
}
}
});
```

This approach ensures that if the backend service is redeployed to a new URL, the frontend will automatically receive the updated value upon its next pulumi up execution.

Operational Workflow and Deployment Ordering

The use of StackReferences introduces a strict temporal dependency in the deployment pipeline. Because a StackReference looks up data at runtime, the target stack must be deployed and its outputs exported before the consuming stack can be initialized.

Deployment Sequencing

In a CI/CD pipeline, the order of operations is critical. If you are deploying a full environment from scratch, the sequence must follow the dependency graph:

1. Deploy the Networking project.
2. Deploy the Cluster project (referencing Networking).
3. Deploy the Backend project (referencing Cluster).
4. Deploy the Frontend project (referencing Backend and Cluster).

If any of these steps are performed out of order, the pulumi up command for the consuming project will fail because the StackReference will be unable to find the required outputs.

Handling Environment Parity

To avoid hardcoding stack names like prod or staging, developers should utilize Pulumi configuration files. By using pulumi.getStack(), the program can dynamically build the reference name based on the current environment.

typescript const stack = pulumi.getStack(); // returns 'dev', 'staging', or 'prod' const stackRef = new pulumi.StackReference(`organization/my-app/${stack}`);

This ensures that the dev application stack references the dev networking stack, and the prod application stack references the prod networking stack, without changing a single line of code.

Summary Comparison of State Access Methods

The following table compares the standard Pulumi state management approach with the StackReference pattern.

Technical Analysis and Conclusion

The Pulumi StackReference is more than a simple utility for sharing variables; it is the fundamental building block for implementing the principle of least privilege and separation of concerns within an infrastructure-as-code ecosystem. By shifting from a monolithic state to a distributed state model, organizations can align their technical architecture with their organizational structure (Conway's Law).

The power of the StackReference lies in its ability to maintain a "single source of truth." There is no need to manually sync IDs or URLs between different configuration files or environment variables. The infrastructure itself becomes the API. When the networking team modifies a subnet, that change propagates naturally to the cluster team, who then propagates it to the application team.

However, the implementation of StackReferences requires a disciplined approach to CI/CD. The temporal dependencies created by these references mean that pipelines must be designed with an understanding of the infrastructure graph. A failure in a base-level stack (e.g., networking) can potentially block all downstream deployments. Therefore, implementing robust testing and validation at each layer of the stack is mandatory.

Ultimately, for any project scaling beyond a few dozen resources or involving more than one team, the transition to a multi-project architecture utilizing StackReferences is inevitable. Whether utilizing the managed Pulumi Cloud for its ease of use or a custom Azure Blob backend for sovereign control, mastering the flow of data between stacks is the key to building resilient, maintainable, and scalable cloud infrastructure.

Sources

1. Martinjt.me - Pulumi Multiple Projects with Custom Backends
2. Daninacan.com - How to Use Pulumi with C# Stack References
3. Pulumi Tutorials - Building with Pulumi: Stack References
4. OneUptime Blog - Pulumi Stack References