Podman Kube Integration and Orchestration

The integration of Podman with Kubernetes YAML manifests represents a fundamental shift in how developers bridge the gap between local development and production orchestration. By utilizing the play kube and generate kube subcommands, Podman provides a seamless pipeline for recording and replaying pods. This capability allows a developer to define a workload in a standard Kubernetes YAML format and execute it locally using Podman's engine, effectively treating the local machine as a lightweight Kubernetes node. The primary objective of this integration is to facilitate the testing of workloads in an environment that mimics Kubernetes as closely as possible before the final deployment into a full-scale cluster. This creates a high-fidelity development loop where the transition from a local container to a managed Kubernetes pod is nearly transparent.

Podman Play Kube and Local Orchestration

The podman play kube subcommand serves as the execution engine for Kubernetes-style YAML files within the Podman ecosystem. It allows users to take a YAML description of a pod—typically one that would be deployed via kubectl create -f—and instantiate it using Podman's internal pod management. This creates a symbiotic relationship between the developer's local environment and the target production environment.

One of the most significant evolutions in this feature is the implementation of on-the-fly image building. In traditional Kubernetes deployments, the image must already exist in a registry. However, users comparing play kube to Docker Compose expressed a need to build images as part of the orchestration process. Podman addressed this by implementing a directory-lookup mechanism. When podman play kube is executed, the engine scans the local file system for a directory that shares the exact same name as the image specified in the YAML file. If such a directory exists and contains either a Containerfile or a Dockerfile, Podman automatically triggers a build process for that image before deploying the pod.

For instance, consider a scenario where a YAML file specifies an image named php-7.2-apache-mysqli:latest. Podman will look for a directory named php-7.2-apache-mysqli. If this directory is found and contains the necessary build instructions, the container image is constructed locally, ensuring that the most recent code changes are incorporated into the pod without requiring a manual podman build step or a push to a remote registry.

Lifecycle Management and the Teardown Process

As the usage of play kube expanded, the need for an efficient teardown mechanism became apparent, particularly for users accustomed to the docker compose down workflow. This led to the introduction of the --down flag in Podman v3.4.

The --down flag implements a comprehensive teardown of the orchestrated environment. When this flag is applied to the podman play kube command, the following sequence occurs:

• Pod Identification: Podman identifies all pods and their constituent containers that were created via the specific YAML manifest.
• Process Termination: All identified pods and containers are stopped.
• Resource Removal: The pods and containers are subsequently removed from the system.

It is important to note that the --down flag does not remove volumes. If a volume is in use, it persists after the teardown, ensuring that data is not lost between different iterations of the pod's lifecycle.

To illustrate this process, consider a pod created from a php.yaml file. Upon execution, the pod might consist of two containers: the actual application container and the pod's infra container (the pause container). A user can verify this status using the following commands:

podman pod ps

This command reveals the POD ID, name, status, and the number of containers. For a PHP example, the output would show the pod as "Running" with 2 containers.

podman ps -a

This provides a detailed view of the individual containers. The output would show the infra container (using an image like k8s.gcr.io/pause:3.5) and the application container (such as localhost/php-7.2-apache-mysqli:latest).

To decommission this environment, the user executes:

podman play kube --down php.yaml

The system then outputs the IDs of the stopped and removed pods, returning the environment to a clean state.

Running Podman Inside Kubernetes Pods

Executing Podman within a Kubernetes pod allows for the creation of complex toolchains where a pod acts as a management node or a build agent. This "Podman-in-Kubernetes" architecture can be achieved through several different configuration paths, depending on the required privilege level and the desired interaction with the host system.

The Remote Socket Leak Method

A powerful technique for integrating Podman into a Kubernetes pod is the leakage of the Podman socket from the host into the container. This allows the container to issue commands to the host's Podman engine, effectively using the pod as a remote control for the host's container resources.

To implement this, the host's Podman socket must be enabled. The Kubernetes YAML configuration (remote.yaml) must define a volume mount that maps the host's socket path to a path within the container.

The following YAML configuration demonstrates this setup:

yaml apiVersion: v1 kind: Pod metadata: name: podman-remote spec: containers: - name: remote image: quay.io/podman/stable args: - sleep - "1000000" volumeMounts: - mountPath: /var/run/podman name: podman-sock volumes: - name: podman-sock hostPath: path: /var/run/podman

Once deployed, a user can enter the pod using kubectl exec -it podman-remote -- sh. Inside the pod, the user can run Podman commands using the --remote flag, which directs the command to the leaked socket. For example:

podman --remote run ubi8 echo hello

This command instructs the host's Podman engine to pull the ubi8 image and execute the echo hello command, with the result being returned to the container's shell. This architecture also supports building images. If a Containerfile is present within the pod, the following command can be used:

podman --remote build -t myimage -f Containerfile .

This process offloads the actual build effort to the host machine while maintaining the management interface within the Kubernetes pod.

Rootless Podman with Chroot Isolation

For scenarios requiring higher security and lower privileges, Podman can be run in a rootless mode inside a Kubernetes pod. This approach avoids the use of the privileged flag and instead relies on host path mounts for container storage.

In this configuration, the pod's YAML defines a volume mount for the container storage directory:

yaml volumeMounts: - mountPath: /home/podman/.local/share/containers name: podman-local volumes: - name: podman-local hostPath: path: /home/umohnani/.local/share/containers

When operating in this mode, the user can perform builds using the --isolation chroot flag. For example:

podman build --isolation chroot -t myimage -f containerfile .

This allows the creation of images within a rootless environment, providing an additional layer of isolation and security by ensuring the process does not have root access to the host.

Rootful Podman Without Privileged Flag

There is a middle ground where Podman runs as root inside the container but without the global privileged: true flag. This requires the precise addition of specific Linux capabilities to the pod's security context.

The following capabilities are mandatory for this configuration:

• CAP_SYS_ADMIN: Necessary for the root Podman instance to mount the required file systems.
• CAP_MKNOD: Required to create the devices in /dev.
• CAP_SYS_CHROOT and CAP_SETFCAP: These are part of the default capabilities required by Podman's internal operations.

The corresponding YAML configuration (rootful-no-priv.yaml) specifies these capabilities as follows:

yaml apiVersion: v1 kind: Pod metadata: name: no-priv-rootful spec: containers: - name: no-priv-rootful image: quay.io/podman/stable args: - sleep - "1000000" securityContext: capabilities: add: - "SYS_ADMIN" - "MKNOD" - "SYS_CHROOT" - "SETFCAP" resources: limits: github.com/fuse: 1

This configuration allows the user to run commands like podman run ubi8 echo hello within the pod, granting the necessary privileges for container management without exposing the host to the risks associated with a fully privileged container.

Comparative Analysis of Podman Kube Integration Methods

The choice between various Podman and Kubernetes integration methods depends heavily on the security requirements and the intended use case. The following table summarizes the key differences.

Detailed Technical Specifications for Podman Manifests

When constructing manifests for podman play kube, the YAML structure must adhere to Kubernetes API standards to ensure compatibility. A typical pod definition includes metadata, specifications for containers, and resource constraints.

In the provided PHP example, the YAML includes the following critical components:

• apiVersion: v1: Specifies the version of the Kubernetes API.
• kind: Pod: Defines the resource type as a Pod.
• metadata: Contains labels (e.g., app: php) and the pod name (php).
• spec: This section defines the operational parameters.
  • containers: A list of containers within the pod.
    • image: The reference to the image (e.g., php-7.2-apache-mysqli:latest).
    • ports: Mapping of container ports to host ports (e.g., containerPort: 80, hostPort: 8080).
    • securityContext: Defines capabilities and privileges (e.g., allowPrivilegeEscalation: true, privileged: false).
    • workingDir: The internal path for the application (e.g., /var/www/html).

The use of capabilities within the securityContext is a vital part of the Podman experience. For instance, dropping CAP_MKNOD, CAP_NET_RAW, and CAP_AUDIT_WRITE enhances the security posture of the container by removing unnecessary privileges that could be exploited.

Analysis of the Podman-to-Kubernetes Pipeline

The ability to move between generate kube and play kube creates a recursive development cycle. generate kube allows a user to export a running Podman pod as a YAML file, which can then be committed to version control. Later, this same file can be instantiated on another machine using play kube.

This workflow effectively replaces the need for complex orchestration scripts during the initial phases of development. By supporting Kubernetes-style init containers, Podman allows developers to prepare the environment—such as downloading configuration files or initializing databases—before the primary application containers start.

The evolution of these tools indicates a trajectory toward complete parity between local container runtimes and cloud-native orchestrators. The ability to build images on the fly and tear down complex pod structures via a single flag (--down) reduces the cognitive load on the developer and minimizes the "it works on my machine" syndrome.

The integration of Podman into Kubernetes pods, whether via remote sockets or specific capability additions, further extends this utility. It enables the creation of "meta-containers" that can manage other containers, creating a tiered architecture suitable for CI/CD pipelines, where a Kubernetes pod acts as the orchestrator for various build and test containers.

Sources

1. Red Hat Blog: Podman play kube updates
2. Red Hat Blog: Podman inside Kubernetes