The implementation of K3s on Raspberry Pi hardware represents a pivotal intersection between edge computing and container orchestration. K3s is a lightweight Kubernetes distribution specifically engineered to operate in resource-constrained environments, making the Raspberry Pi ecosystem—ranging from the legacy Raspberry Pi 3 and 4 to the high-performance Raspberry Pi 5—an ideal substrate for these workloads. By stripping out legacy cloud-provider code and optimizing the binary, K3s allows users to deploy a fully functional Kubernetes control plane and worker node architecture on single-board computers (SBCs). This capability enables the creation of home labs, edge computing clusters, and development environments that mirror production-grade Kubernetes architectures without requiring expensive, power-hungry server hardware.
The utility of K3s on Raspberry Pi extends beyond simple experimentation. For technical enthusiasts and DevOps professionals, it provides a tangible environment to test Microservices Architecture, implement CI/CD pipelines using tools like GitHub Actions or GitLab CI, and explore the complexities of distributed systems. Whether utilizing a Turing Pi cluster for high-density compute or individual boards connected via a network switch, the deployment of K3s transforms a collection of ARM-based devices into a cohesive, scalable compute fabric.
Hardware Specifications and Infrastructure Requirements
The hardware selection for a K3s cluster determines the ceiling of the workload capacity and the overall stability of the orchestration layer. While K3s is designed for efficiency, the underlying physical components must meet specific thresholds to prevent catastrophic failure under load.
The Raspberry Pi 5 is positioned as a powerhouse for these deployments. To ensure stable operation, a minimum of 4GB of RAM is required, although 8GB is strongly recommended for larger workloads. This RAM overhead is critical because Kubernetes components, including the kubelet and the container runtime, consume significant memory. Insufficient RAM leads to Out-of-Memory (OOM) kills, which can destabilize the control plane and lead to frequent pod restarts.
Storage is another critical vector for system reliability. While a high-quality microSD card of 32GB or larger is a common starting point, it is often insufficient for production-like environments. The etcd database, which serves as the source of truth for the Kubernetes cluster, is extremely write-intensive. MicroSD cards and eMMC storage typically cannot handle the continuous IO load associated with etcd, leading to premature disk failure or severe performance degradation. Consequently, the use of an external SSD via USB is highly recommended to ensure optimal speed and longevity.
For those building larger clusters, the following server sizing guide outlines the relationship between control-plane resources and agent scalability:
| Server CPU | Server RAM | Number of Agents |
|---|---|---|
| 2 | 4 GB | 0-350 |
| 4 | 8 GB | 351-900 |
| 8 | 16 GB | 901-1800 |
| 16+ | 32 GB | 1800+ |
When implementing a high-availability (HA) setup consisting of three server nodes, the scalability increases by roughly 50%. For example, a three-server configuration with 4 vCPU and 8GB RAM can scale to approximately 1200 agents.
System Preparation and OS Configuration
Before the K3s binary can be executed, the host operating system must be tuned to support the requirements of the Kubernetes runtime. Raspberry Pi OS (64-bit) is the recommended distribution, as it allows the system to leverage the full 64-bit instruction set of the ARM processor, improving both performance and memory addressing.
The initial preparation phase involves updating the system packages to ensure the latest security patches and dependencies are present. This is achieved through the following commands:
sudo apt update -y
Following the update, essential networking and transfer tools must be installed to facilitate the downloading of the K3s installation script:
sudo apt install -y curl wget
A critical technical requirement for Kubernetes is cgroup (control group) support. Cgroups are used by the Linux kernel to limit, account for, and isolate the resource usage (CPU, memory, disk I/O) of process groups. Without memory cgroups enabled, K3s will fail to manage container resources effectively, leading to cluster instability.
To enable cgroup support, the /boot/firmware/cmdline.txt file must be edited. This file contains the kernel boot parameters.
sudo nano /boot/firmware/cmdline.txt
The following parameters must be appended to the end of the existing line, ensuring that all parameters remain on a single line:
cgroup_memory=1 cgroup_enable=memory
After saving the file, a system reboot is mandatory to apply the kernel changes:
sudo reboot
K3s Installation and Control Plane Setup
Once the hardware is prepared and the kernel is configured for cgroups, the installation of K3s is performed using a streamlined installation script. This script automates the downloading of the binary, the configuration of systemd services, and the initialization of the Kubernetes environment.
The installation is triggered by executing the following command:
curl -sfL https://get.k3s.io | sh -
By default, this command installs K3s as a server, which encompasses both the control plane (the "brains" of the cluster) and a single-node worker. This results in a functional, single-node Kubernetes cluster immediately upon completion. To verify that the installation was successful and the service is operating correctly, the status of the K3s systemd unit should be checked:
sudo systemctl status k3s
If the service is active and running, the operator can begin interacting with the cluster. K3s installs kubectl, the standard Kubernetes command-line tool, by default. To verify the status of the node and ensure it has reached the Ready state, the following command is used:
sudo kubectl get nodes
Scaling the Cluster: Adding Worker Nodes
A single-node cluster is useful for testing, but the true power of Kubernetes lies in distributed computing. Expanding the cluster involves adding worker nodes (agents) that connect back to the primary server.
To join a worker node to the cluster, the operator first needs the unique node token from the server. This token acts as a shared secret to authenticate the worker node during the join process. The token is retrieved using:
sudo cat /var/lib/rancher/k3s/server/node-token
Once the token is obtained, the installation script must be run on the worker node. This command requires the URL of the server (usually on port 6443) and the token. The command is structured as follows:
curl -sfL https://get.k3s.io | K3S_URL=https://<SERVER_IP>:6443 K3S_TOKEN=<NODE_TOKEN> sh -
Replacing <SERVER_IP> and <NODE_TOKEN> with the actual values allows the worker node to register itself with the control plane.
Advanced Deployment and Infrastructure Management
For those managing larger fleets of Raspberry Pis, manual installation is inefficient. Automated configuration management tools like Ansible allow for the simultaneous deployment of K3s across multiple nodes.
The deployment process using k3s-ansible involves several structured steps:
- Installation of Ansible via pip3:
pip3 install ansible --user
export PATH="/Users/bbende/Library/Python/3.7/bin:$PATH"
- Cloning the
k3s-ansiblerepository and preparing the inventory. The inventory file (hosts.ini) defines the architecture of the cluster. An example configuration for a three-node cluster is as follows:
[master] 192.168.1.244
[node] 192.168.1.245 192.168.1.246
[k3s_cluster:children] master node
- Specifying the K3s version in the
group_vars/all.ymlfile:
k3s_version: v1.21.0+k3s1
- Executing the Ansible playbook to automate the installation:
ansible-playbook site.yml -i inventory/rpi/hosts.ini
Remote Cluster Management and Kubectl Configuration
Interacting with a Raspberry Pi cluster from a local workstation is more efficient than SSHing into individual nodes. This requires transferring the Kubeconfig file from the master node to the local machine.
The Kubeconfig file is transferred using secure copy (SCP):
scp [email protected]:~/.kube/config ~/.kube/config-rpi-k3s
To ensure the local kubectl client recognizes the cluster, the KUBECONFIG environment variable must be updated in the shell configuration file (e.g., .zshrc):
export KUBECONFIG="~/.kube/config"
export KUBECONFIG="$KUBECONFIG:~/.kube/config-rpi-k3s"
Once configured, the available contexts can be viewed to confirm the connection:
kubectl config get-contexts
If the context name is not descriptive, it can be renamed for better organization:
kubectl config rename-context <CURRENT_NAME> <NEW_NAME>
The final verification of the cluster's health is performed by listing the nodes and their status:
kubectl --context rpi-k3s get nodes
Network Architecture and Connectivity
A functional K3s cluster relies on specific network ports being open and accessible across all nodes. Failure to configure these ports leads to communication breakdowns between the control plane and the worker nodes.
The K3s server requires port 6443 to be accessible by all nodes in the cluster. This port is the primary entry point for the Kubernetes API.
For node-to-node communication, K3s utilizes Flannel as the default CNI (Container Network Interface). The port requirements vary based on the backend used:
- Flannel VXLAN backend: requires UDP port
8472. - Flannel WireGuard backend: requires UDP port
51820, and UDP port51821if IPv6 is enabled.
The nodes should not listen on any other ports unless specifically required by the deployed applications.
Analysis of Cluster Performance and Stability
The transition from a single-node setup to a multi-node Raspberry Pi cluster introduces a complex set of trade-offs. The primary bottleneck in these environments is almost always I/O throughput. As noted previously, the reliance on SD cards for etcd storage is a critical failure point. In a production-style environment, the write-heavy nature of the etcd database creates an IO wait state that can lead to "node not ready" statuses, even if the CPU and RAM usage are low. Transitioning to USB-attached SSDs effectively mitigates this risk, transforming the cluster from a fragile experiment into a robust compute platform.
Another critical consideration is the "join spike." When new agent nodes join a cluster, there is a significant spike in CPU utilization on the server node. To prevent the control plane from becoming unresponsive, it is recommended to join agent nodes in batches of 50 or fewer. This staggered approach allows the CPU to stabilize between batches.
Furthermore, network stability is paramount. While Raspberry Pis offer on-board Wi-Fi, a wired Ethernet connection via a high-quality switch is strongly recommended. Network latency and packet loss over Wi-Fi can cause the Kubelet to report the node as Unhealthy, triggering unnecessary pod evictions. Using a Turing Pi cluster case or similar hardware further stabilizes the environment by providing standardized power and networking paths.
Finally, the choice of OS—specifically 64-bit vs 32-bit—impacts the memory ceiling. In a 32-bit environment, individual processes are limited in the amount of RAM they can address. For a 4GB or 8GB Raspberry Pi, 64-bit OS is mandatory to ensure the Kubernetes components can utilize the available hardware effectively.