Raspberry Pi 4 K3S Cluster Architecture and RancherOS Deployment

The convergence of ARMv8 architecture and container orchestration has enabled the creation of highly portable, energy-efficient Kubernetes environments. By leveraging the Raspberry Pi 4, developers and systems engineers can construct a physical cluster that mirrors the operational logic of cloud-native environments while remaining contained within a compact, hardware-defined footprint. The integration of Rancher K3S allows for a lightweight distribution of Kubernetes, optimized specifically for resource-constrained environments, effectively stripping away the bloat of standard Kubernetes (K8s) to ensure that the majority of the system's 8GB of RAM is allocated to the workloads rather than the orchestration overhead.

This architecture transforms a set of single-board computers into a distributed system capable of high availability. When deployed using a rack tower configuration, the cluster moves beyond a mere hobbyist project and becomes a portable server unit. The implementation of Power over Ethernet (PoE) further simplifies the physical infrastructure, reducing the cabling requirements to a single network cable per node, which provides both data connectivity and electrical power. This consolidation is critical for portability, allowing the entire cluster to be powered by a single PoE switch, thereby eliminating the need for multiple individual power bricks that would otherwise clutter the deployment.

Hardware Specifications and Physical Infrastructure

The physical foundation of a portable Kubernetes cluster requires a precise selection of components to ensure stability, power delivery, and sufficient storage. The use of the Raspberry Pi 4 Model B is central to this design due to its ARMv8 processor and increased memory capacity, which is essential for running the K3S control plane and worker nodes.

The following table details the specific hardware components utilized in the construction of a 4-node portable cluster:

Part Quantity Unit Cost (CHF) Specifications
Raspberry Pi 4 Model B 4 89.90 8GB RAM, ARMv8 Architecture
Verbatim Nano USB Sticks 4 11.95 64GB Capacity
52Pi ZP-0088 Rack Tower 1 35.00 Hardware Housing
Tp-Link TL-SG1005P 1 51.30 POE Switch
LogiLink POE Cables 4 (Not Specified) U/FTP Category

The 52Pi ZP-0088 Rack Tower provides the structural integrity required for a portable unit. It features a handle designed for transport, which allows the operator to move the entire cluster between environments. However, the assembly of the case presents challenges regarding accessibility. Specifically, the card slots become difficult to reach once the case is fully assembled. To mitigate this, a custom modification involving a wooden plate with pre-drilled holes is employed. This plate is secured to the base of the rack tower using four blue cable ties. This modification ensures that the hardware remains accessible for maintenance.

The network and power backbone is handled by the Tp-Link TL-SG1005P POE Switch. This device is connected to the rack tower via two screws utilizing the wall-mount recess of the switch. The use of LogiLink U/FTP POE cables allows for a streamlined cabling approach where each Raspberry Pi receives power and network connectivity through a single cable. There is an ongoing consideration regarding the addition of a mobile router to the assembly; however, the primary concern is whether the POE switch can deliver the additional power required to sustain a router in addition to the four Pi nodes.

RancherOS Installation and Configuration

RancherOS serves as a specialized operating system designed to run containers as the primary mechanism for system services. As of version 0.5.0, RancherOS provides a dedicated Raspberry Pi image. This image is distributed via the official releases page and must be flashed onto the SD card following the standard Raspberry Pi documentation.

A critical limitation of the RancherOS installation process on the Raspberry Pi is the inability to pass in a cloud-config during the initial installation phase. This means that the system must be booted in its default state, the configuration must be manually adjusted, and the system must then be rebooted to apply the changes. It is important to note that the ros install command is not necessary after the RancherOS image has been flashed to the SD card.

The system utilizes a specific approach to partition management. RancherOS does not automatically expand the root partition to fill the entire capacity of the SD card. This limitation can lead to inefficient use of storage, especially when using high-capacity cards. To address this, a manual workaround is required to store Docker containers on a larger partition that fills the remainder of the disk.

The process for expanding the Docker root involves the following technical steps:

  • Initialize the partition tool using sudo fdisk /dev/mmcblk0.
  • Create a new partition by selecting n, pressing [Enter] four times to accept default values, and then writing the table by selecting w.
  • Reboot the system to reload the new partition tables using sudo reboot.
  • Create the target directory for the Docker root using sudo mkdir /mnt/docker.
  • Configure Docker to use the new root via the command sudo ros config set rancher.docker.extra_args [-g,/mnt/docker].
  • Format the new partition to ext4 using sudo mkfs.ext4 /dev/mmcblk0p3.
  • Ensure the mount persists after reboot by executing sudo ros config set mounts "[['/dev/mmcblk0p3','/mnt/docker','ext4','']]".
  • Mount the Docker root using sudo mount /dev/mmcblk0p3 /mnt/docker.
  • Restart the Docker service using sudo system-docker restart docker.

If the installation is not new, the existing Docker root located at /var/lib/docker must be migrated to the new root. This is achieved by recursively copying all files using sudo cp -R /var/lib/docker/* /mnt/docker, followed by another restart of the Docker service with sudo system-docker restart docker.

Network Configuration and Connectivity

Connectivity is the most critical component of a Kubernetes cluster, as nodes must maintain constant communication. While the primary configuration relies on POE and ethernet for stability, RancherOS (starting from v1.5.2) supports Wi-Fi connectivity.

To enable Wi-Fi on a Raspberry Pi running RancherOS, the following command sequence is utilized:

  • Load the wireless driver using modprobe brcmfmac.
  • Generate the wpa_supplicant configuration using wpa_passphrase <ssid> <psk> > /etc/wpa_supplicant.conf.
  • Initiate the supplicant using wpa_supplicant -iwlan0 -B -c /etc/wpa_supplicant.conf.
  • Assign an IP address using dhcpcd -MA4 wlan0.

Alternatively, if the user has access to a cloud-config, Wi-Fi can be enabled using the following configuration block:

```yaml

cloud-config

rancher:
network:
interfaces:
wlan0:
wifinetwork: network1
wifi
networks:
network1:
ssid: "Your wifi ssid"
psk: "Your wifi password"
scan_ssid: 1
```

A known issue with this configuration is that the Raspberry Pi may automatically drop the Wi-Fi connection after a period of time. This behavior is attributed to the system's power management settings, which can interfere with the stability of the Kubernetes cluster.

For environments utilizing a home router, such as the Mikrotik RB4011, the LAN CIDR is typically configured (e.g., 192.168.11.0/24). To expose Kubernetes services to the external internet, dst-nat rules must be configured via script. For example, the IP 192.168.11.50 can be designated as the Loadbalancer IP to route external traffic to the internal cluster services.

Storage Architecture and Redundancy

A functional Kubernetes cluster requires a robust storage solution to handle persistent volumes. Relying solely on the SD card is insufficient for production-like workloads due to the limited write endurance of flash memory and the lack of redundancy.

To solve this, the architecture incorporates a storage cluster using 64GB Verbatim Nano USB sticks. The goal is to implement a four-times redundant Gluster volume. This setup ensures that if one USB stick or one node fails, the data remains available across the other nodes.

The storage stack is implemented through several layers:

  • The installation of a storage class and an NFS provisioner to allow Kubernetes to dynamically request storage.
  • The deployment of Gluster volumes using the USB sticks as the underlying storage medium.
  • The integration of Heketi, which allows for the automatic provisioning of Gluster volumes, removing the need for manual volume creation.

This layered approach ensures that the portable cluster can handle stateful applications, as the storage is distributed across the four Raspberry Pi nodes rather than being localized to a single device.

Comparison of Deployment Environments

Depending on the user's goals, the deployment of a Raspberry Pi cluster can vary. While some utilize RancherOS for its container-first approach, others prefer standard distributions like Ubuntu 20.04.

The following table compares the two primary deployment approaches found in the reference data:

Feature RancherOS Approach Ubuntu 20.04 Approach
Base OS RancherOS (v0.5.0+) Ubuntu 20.04
Memory 8GB (Pi 4) 4GB (Pi 4)
Storage SD Card + USB Sticks 64GB Samsung SDXC
Network POE Switch / Wi-Fi POE Headers / Home Router
Orchestration Rancher K3S Kubernetes (Small Cluster)
Configuration Manual post-boot / cloud-config Standard Ubuntu config

The Ubuntu-based approach often utilizes a smaller memory footprint per node (4GB) and relies on a standard home router for connectivity. In contrast, the RancherOS approach is more focused on the "cluster-as-an-appliance" model, utilizing 8GB nodes and a rack-mounted POE switch for maximum portability.

Analysis of Technical Challenges and Solutions

The deployment of a portable Kubernetes cluster on Raspberry Pi 4 introduces several technical friction points that require specific engineering solutions.

The most prominent challenge is the power management of the hardware. When using POE, the power budget of the switch becomes the limiting factor. The Tp-Link TL-SG1005P provides the necessary wattage for the Pi nodes, but the addition of peripheral devices, such as mobile routers, requires a careful calculation of the POE power budget to avoid system instability or brownouts.

Storage management presents the second major challenge. The default behavior of RancherOS not expanding the root partition creates a discrepancy between the physical capacity of the SD card and the available logical space. The solution implemented—creating a separate partition for Docker and mounting it via ros config—effectively bypasses this limitation. By moving the Docker root to /mnt/docker, the system can utilize the full capacity of the SD card for container images and volumes.

Furthermore, the physical assembly of the 52Pi Rack Tower demonstrates the need for custom modifications. The inability to access card slots after assembly is a common issue in compact enclosures. The use of a wooden plate and cable ties is a pragmatic solution to an engineering oversight in the case design, ensuring that the hardware can be serviced without complete disassembly.

Finally, the issue of Wi-Fi instability on the Raspberry Pi, caused by power management, highlights the necessity of wired connections in a Kubernetes environment. The network requirements for K3S—specifically the need for low latency and high reliability for the etcd store and API server—make the POE ethernet approach significantly superior to any Wi-Fi-based alternative.

Conclusion

The creation of a portable Kubernetes cluster using Raspberry Pi 4 and Rancher K3S represents a sophisticated intersection of hardware engineering and software orchestration. By utilizing 8GB ARMv8 nodes, a POE-powered network, and a distributed Gluster storage system, it is possible to build a system that is both physically portable and computationally capable. The transition from a standard SD card boot to a partitioned Docker root ensures that the storage capacity is maximized, while the use of Heketi and Gluster provides the redundancy necessary for stateful workloads.

This architecture proves that the constraints of single-board computers can be overcome through strategic configuration. The shift toward POE cabling reduces the physical complexity of the cluster, and the use of a rack tower provides the structural stability needed for movement. While challenges such as Wi-Fi power management and case accessibility persist, the implementation of manual workarounds and custom hardware modifications allows for a stable, production-ready environment. Ultimately, this setup provides a scalable blueprint for anyone seeking to implement a local "cloud@home" project that balances portability with the full power of the Kubernetes ecosystem.

Sources

  1. Blog Loitzl - Portable Kubernetes Cluster
  2. Rancher Documentation - Raspberry Pi Installation
  3. Rancher Documentation - Running RancherOS
  4. GitHub Gist - Kubernetes Raspberry Pi 4 Setup

Related Posts