Podman Kubernetes Integration and Log Driver Configuration

Published by TechMarmot Editorail Staff on | Category Podman

The architectural convergence of Podman and Kubernetes represents a strategic shift in container orchestration and runtime management. Podman, designed as a lightweight container engine, provides a command-line interface that serves as a drop-in replacement for Docker, specifically maintaining compatibility with the Docker CLI while excluding Docker Swarm commands. Beyond mere compatibility, Podman possesses an intrinsic ability to parse and generate Kubernetes manifests. This integration allows for a fluid transition of workloads, enabling developers to manage Podman workloads within a Kubernetes cluster or transfer existing Kubernetes workloads to a standalone Podman installation with minimal friction. The synergy between these two technologies allows organizations to leverage the agility of a daemonless container engine alongside the robust orchestration capabilities of a production-grade cluster.

Rootful Podman Execution Without Privileged Flags

Operating Podman in a rootful capacity within a Kubernetes pod without utilizing the global privileged flag requires a surgical application of Linux capabilities. When Podman is executed, it dynamically adds the capabilities it requires to function. If a Kubernetes pod is launched without these specific capabilities, the Podman process will encounter an immediate failure.

To achieve rootful execution, the securityContext of the pod must be explicitly configured to add specific capabilities. The primary requirements include SYS_ADMIN, MKNOD, SYS_CHROOT, and SETFCAP.

  • SYS_ADMIN: This capability is mandatory for Podman running as root inside a container to mount the necessary file systems. Without SYS_ADMIN, the container cannot perform the mount operations required to isolate the containerized environment.
  • MKNOD: This is required for Podman to create the essential devices in /dev. Failure to include MKNOD prevents the creation of character and block devices that containers may rely on for hardware interaction.
  • SYS_CHROOT: This capability is part of the default list of requirements for Podman. It allows the process to change the root directory for the current process and its children, which is fundamental to container isolation.
  • SETFCAP: This is required to set and modify file capabilities. Like SYS_CHROOT, it is part of the default Podman capability list and must be granted by the Kubernetes orchestrator to prevent execution failure.

The implementation of this configuration is detailed in the following YAML structure:

yaml apiVersion: v1 kind: Pod metadata: name: no-priv-rootful spec: containers: - name: no-priv-rootful image: quay.io/podman/stable args: - sleep - "1000000" securityContext: capabilities: add: - "SYS_ADMIN" - "MKNOD" - "SYS_CHROOT" - "SETFCAP" resources: limits: github.com/fuse: 1

Under this configuration, a user executing kubectl exec -it no-priv-rootful -- sh will find themselves with root privileges (uid=0(root) gid=0(root) groups=0(root)). This allows for the successful execution of container commands, such as podman run ubi8 echo hello. During this process, Podman resolves "ubi8" as an alias via /etc/containers/registries.conf.d/000-shortnames.conf, pulls the image from registry.access.redhat.com/ubi8:latest, and outputs the expected string.

Podman-Remote Orchestration via Host Socket Leaking

A more advanced integration pattern involves the use of podman-remote, where the Podman socket resides on the host machine and is leaked into the Kubernetes pod. This architecture offloads the container runtime overhead to the host while allowing the pod to act as a remote controller.

To implement this use case, two prerequisite environmental conditions must be met on the host:

  • Disable SELinux on the host to prevent security policy interference with the socket communication.
  • Enable the Podman socket on the host to allow remote connections.

The configuration requires a volume mount that maps the host's Podman socket into the container's filesystem. The following YAML defines this relationship:

yaml apiVersion: v1 kind: Pod metadata: name: podman-remote spec: containers: - name: remote image: quay.io/podman/stable args: - sleep - "1000000" volumeMounts: - mountPath: /var/run/podman name: podman-sock volumes: - name: podman-sock hostPath: path: /var/run/podman

By mounting /var/run/podman from the host to the same path in the pod, the container gains access to the host's container engine. When a user executes podman --remote run ubi8 echo hello inside this pod, the command is proxied to the host. This enables the Podman client inside the container to trigger image pulls and container execution on the host system, as evidenced by the retrieval of image blobs and configuration from the registry.

Furthermore, this "leaked socket" approach supports build operations. A user can create a Containerfile within the pod and execute:

bash podman --remote build -t myimage -f Containerfile .

This triggers the build process on the host, allowing for the installation of packages (e.g., dnf install -y busybox in a Fedora image) and the setting of environment variables (e.g., ENV foo=bar), all while the control plane remains within the Kubernetes pod.

Comprehensive Analysis of Podman Log Drivers

Podman provides a diverse array of log drivers that allow users to control where container logs are stored and how they are formatted. The selection of a log driver impacts disk I/O, log accessibility, and integration with system-level monitoring tools.

Log Driver Comparison and Specifications

The following table delineates the characteristics and primary use cases for each available log driver:

Log Driver Storage Method Primary Use Case Key Feature
k8s-file Plain text files Podman-generated K8s YAML Accessible via podman logs
journald systemd journal Centralized system logging Structured metadata (ID, Name)
json-file Kubernetes format Docker-compatible scripting Support for max-size limits
passthrough No capture Interactive containers Minimal overhead
none No storage High-volume, unneeded logs Minimal disk I/O

Deep Dive into Driver Implementation

Each log driver requires specific flags during the podman run command to be activated.

  • k8s-file: This driver stores logs as plain text. It is the recommended choice when running Podman-generated Kubernetes YAML.

    Example implementation:
    bash podman run -d --log-driver k8s-file --name test-k8s alpine sh -c 'echo "hello from k8s-file"'
    Verification can be performed using podman logs test-k8s or by inspecting the host configuration path:
    bash podman inspect --format '{{.HostConfig.LogConfig.Path}}' test-k8s

  • journald: This is the default driver. It redirects logs to the systemd journal, enabling the use of journalctl for querying logs based on container names.

    Example implementation:
    bash podman run -d --log-driver journald --name test-journald alpine sh -c 'echo "hello from journald"'
    Logs are retrieved via:
    bash journalctl CONTAINER_NAME=test-journald --no-pager

  • json-file: This driver serves as an alias for k8s-file, ensuring that logs utilize the Kubernetes log format. This is critical for users who rely on scripts designed for Docker's json-file driver.

    Example implementation:
    bash podman run -d --log-driver json-file --name test-json alpine sh -c 'echo "hello from json-file"'

  • passthrough: This mode allows the application to handle its own logging without Podman intercepting the output. It is optimal for interactive sessions where overhead must be minimized.

    Example implementation:
    bash podman run -d --log-driver passthrough --name my-app my-image:latest

  • none: This driver completely disables log storage. While output remains visible during interactive sessions, nothing is written to disk, which is essential for containers generating excessive, unnecessary logs.

    Example implementation:
    bash podman run -d --log-driver none --name my-app my-image:latest

Advanced Configuration via Log Options

Podman allows for granular control over drivers using the --log-opt flag. This is particularly useful for managing disk space and metadata.

  • Max-size Limits: For json-file and k8s-file drivers, users can prevent disk exhaustion by setting a maximum size.

    Example for json-file:
    bash podman run -d \ --log-driver json-file \ --log-opt max-size=10m \ --name my-app my-image:latest

    Example for k8s-file:
    bash podman run -d \ --log-driver k8s-file \ --log-opt max-size=50m \ --name my-app my-image:latest

  • Custom Tagging: For the journald driver, custom tags can be applied to improve log searchability.

    Example for journald:
    bash podman run -d \ --log-driver journald \ --log-opt tag="{{.Name}}" \ --name my-app my-image:latest

Integration with Podman Compose

Log driver configurations can be codified within a podman-compose.yml file to ensure consistency across services.

yaml services: web: image: nginx:latest logging: driver: journald options: tag: "{{.Name}}" api: image: my-api:latest logging: driver: json-file options: max-size: "10m" worker: image: my-worker:latest logging: driver: k8s-file options: max-size: "50m"

System-Wide Default Configuration

To avoid specifying the log driver for every container, the default can be changed in the containers.conf file. The location of this file depends on the user's privilege level:

  • Rootless users: ~/.config/containers/containers.conf
  • Root users: /etc/containers/containers.conf

To set the default driver to journald for a rootless user, the following steps are taken:

bash mkdir -p ~/.config/containers cat >> ~/.config/containers/containers.conf << 'EOF' [containers] log_driver = "journald" EOF

The current default can be verified with the following command:

bash podman info --format '{{.Host.LogDriver}}'

Technical Analysis of Deployment Paradigms

The integration of Podman into Kubernetes environments facilitates three distinct operational modes, each with specific implications for security and performance.

The first mode is the non-privileged rootful execution. This requires a precise set of capabilities (SYS_ADMIN, MKNOD, SYS_CHROOT, SETFCAP) and specifically involves the use of FUSE limits (github.com/fuse: 1). The consequence of this setup is that Podman can operate as root within the container, allowing for the execution of standard container tasks while remaining within the constraints of a Kubernetes pod. This is verified by the ability to run podman run ubi8 echo hello within the no-priv-rootful pod.

The second mode is the rootless execution. In this scenario, the pod is configured with a non-root user (e.g., uid=1000(podman)). To maintain state and image persistence, a volume mount is used to map the host's container storage to the container's internal path:

  • Mount Path: /home/podman/.local/share/containers
  • Host Path: /home/umohnani/.local/share/containers

Under this rootless configuration, Podman can still pull images and build new images using isolation techniques like chroot. For example, the command podman build --isolation chroot -t myimage -f containerfile . allows for the creation of a local image (localhost/myimage) without requiring administrative privileges on the host.

The third mode is the remote socket connection. This is the most permissive in terms of host interaction but requires the most significant host-side changes (disabling SELinux and enabling the socket). By leaking /var/run/podman into the pod, the pod becomes a thin client. This is highly efficient for environments where multiple pods need to orchestrate containers on a single shared host without each pod requiring its own full container runtime stack.

Conclusion

The convergence of Podman and Kubernetes creates a powerful hybrid environment that bridges the gap between local development and cluster orchestration. By leveraging capabilities like SYS_ADMIN and MKNOD, users can execute rootful Podman within Kubernetes pods without the security risks associated with the global privileged flag. Simultaneously, the ability to leak the Podman socket via hostPath volumes allows for a remote orchestration model that is both flexible and high-performing.

The log driver ecosystem further enhances this integration by providing tailored options for every operational requirement. Whether using journald for system-level auditing, k8s-file for Kubernetes manifest compatibility, or none for high-throughput applications, Podman ensures that log management does not become a bottleneck. The ability to configure these settings via containers.conf or podman-compose.yml ensures that these configurations are reproducible and scalable.

Ultimately, the integration of Podman within Kubernetes allows for a seamless lifecycle—from building images with chroot isolation to deploying them via Kubernetes manifests and monitoring them through diverse log drivers. This architecture minimizes the friction between the developer's local environment and the production cluster, creating a unified container management experience.

Sources

  1. Red Hat Blog
  2. OneUptime
  3. Better Stack

Related Posts