K3s on Raspberry Pi

The convergence of Raspberry Pi hardware and K3s software transforms a collection of credit-card-sized computers into a fully functional Kubernetes cluster. This architecture represents a pivotal shift in the accessibility of cloud-native computing, allowing users to transition from educational toys to legitimate computing platforms. By utilizing K3s, a lightweight Kubernetes distribution, an operator can construct a home lab cluster for pennies on the dollar compared to the prohibitive costs of public cloud alternatives. This setup is not merely an academic exercise; it is a viable foundation for learning Kubernetes orchestration, implementing sophisticated home automation, or deploying edge computing workloads where physical proximity to data sources is critical.

Hardware Architecture and Resource Requirements

The physical foundation of a K3s cluster must be carefully selected to avoid the instability and performance degradation common in resource-constrained environments. While the Raspberry Pi is powerful, the overhead of container orchestration requires a strategic approach to hardware selection.

Minimum Single-Node Configuration

For those initiating a basic setup with a single node, the following minimum specifications are required to ensure the system remains operational.

The shift from 2GB to 4GB or 8GB of RAM is not merely a luxury; it provides the necessary breathing room for the K3s control plane and the actual application workloads. Storage is a critical bottleneck; while 16GB is the absolute minimum, a 32GB A2-rated microSD card is recommended because the A2 rating ensures faster random I/O, which is essential for the frequent read/write operations of a Kubernetes database. For production-grade workloads, booting from an SSD via USB 3.0 is the gold standard to eliminate the latency and wear-and-tear associated with SD cards.

Multi-Node Cluster Strategy

To achieve a true cluster experience, a three-node architecture is recommended. This configuration prevents a single point of failure and allows for the distribution of workloads across multiple physical machines.

• Master Node: A Raspberry Pi 4 with 4GB of RAM acting as the Control Plane.
• Worker Node 1: A Raspberry Pi 4 with 4GB of RAM.
• Worker Node 2: A Raspberry Pi 4 with 4GB of RAM.

The architectural flow involves a home router or switch connecting all nodes. The Master node manages the cluster and communicates with the worker nodes via the K3s Agent.

Cluster Shopping List and Component Impact

Selecting the correct components prevents catastrophic system failure and ensures long-term stability.

• Raspberry Pi 4 (4GB recommended) x3
  • 4GB provides essential breathing room for system workloads.
  • 8GB is necessary if the cluster will host heavy applications or databases.
• microSD cards (32GB A2 rated) x3
  • A2 rating ensures higher random I/O performance.
  • SSD boot is suggested for any production-level workloads.
• USB-C power supplies (5V 3A) x3
  • Underpowered supplies lead to random system crashes and instability.
  • Only official or certified PSUs should be utilized.
• Ethernet cables (Cat6) x3
  • Gigabit networking is essential for efficient cluster traffic.
  • WiFi is discouraged due to latency and reliability issues.

Initial System Configuration

Before the installation of K3s, the underlying operating system must be hardened and configured. This stage ensures that the nodes can communicate reliably and that the system is optimized for Kubernetes.

OS Selection and Basic Setup

The recommended software foundation is Raspberry Pi OS (64-bit). A 64-bit architecture is critical for maximizing the utility of the ARM processor and ensuring compatibility with modern container images.

Initial setup steps include:

• Set hostname: Every node must have a unique identity. For example, use pi-master for the control plane and pi-worker-1, pi-worker-2 for workers.
• Enable SSH: Access must be configured via password or key authentication, with key-based authentication being the recommended standard for enhanced security.
• User Management: Since the default pi user is now deprecated, it is necessary to create a custom user, such as k3s-admin.
• WiFi Configuration: If Ethernet is unavailable, the SSID and password must be entered, and the country code set for regulatory compliance.
• Locale and Timezone: These must be correctly configured to ensure that logs across the cluster have accurate and synchronized timestamps.

System Updates and Tool Installation

Once the OS is booted, the system must be updated to the latest security patches.

• SSH into the Pi:
  ssh [email protected]
• Update system packages:
  sudo apt update && sudo apt upgrade -y
• Install essential tools:
  sudo apt install -y curl wget

Network Stability and Static IP Configuration

A Kubernetes cluster cannot function if nodes frequently change IP addresses. Therefore, static IP addresses are mandatory for cluster stability.

• Edit the dhcpcd configuration:
  sudo nano /etc/dhcpcd.conf

The following configuration should be applied to the end of the file, adjusted for the specific node:

• Master Node:
  interface eth0
static ip_address=192.168.1.100/24
static routers=192.168.1.1
static domain_name_servers=192.168.1.1 8.8.8.8

• Worker Node 1:
  static ip_address=192.168.1.101/24

• Worker Node 2:
  static ip_address=192.168.1.102/24

After editing the file, the system must be rebooted:
sudo reboot

Configuring cgroups for Kubernetes

Kubernetes requires cgroup (control group) support to manage resource allocation for containers. By default, this is not enabled on Raspberry Pi OS, and failing to enable it will cause the K3s service to fail.

• Edit the cmdline.txt file:
  sudo nano /boot/firmware/cmdline.txt

• Append the following parameters to the end of the existing line. It is critical that these remain on the same line:
  cgroup_memory=1 cgroup_enable=memory

• Save the file and reboot the device:
  sudo reboot

K3s Installation and Control Plane Setup

K3s is designed for fast installation on resource-limited devices. The installation process converts a standard Linux environment into a Kubernetes control plane.

The Installation Process

From the main computer, SSH into the Pi designated as the control plane (master). The installation is performed via a single-line script:

curl -sfL https://get.k3s.io | sh -

This command triggers several automated actions:
- It downloads the K3s binary.
- It configures the k3s.service systemd unit.
- It installs kubectl, the Kubernetes command-line tool.
- It initializes the cluster as a server (control plane), creating a single-node Kubernetes cluster by default.

Verification and Service Management

Once the script completes, the operator must verify that the K3s service is active and healthy.

• Check service status:
  sudo systemctl status k3s

• Verify node status via kubectl:
  sudo kubectl get nodes

The output should show the Raspberry Pi listed in a Ready state.

Optimizing kubectl Access

To avoid prefixing every command with sudo or k3s, the K3s kubeconfig must be mapped to the user's home directory.

• Create the .kube directory:
  mkdir -p ~/.kube
• Copy the configuration file:
  sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
• Change ownership to the current user:
  sudo chown $(id-u):$(id-g) ~/.kube/config

With these changes, the operator can simply run:
kubectl get nodes

Advanced K3s Configuration and Customization

K3s provides several environment variables and flags to tailor the distribution for specific hardware constraints or software preferences.

Custom Installation Options

The installation script can be modified using INSTALL_K3S_EXEC to disable specific built-in components or specify versions.

• Disable Traefik (if using nginx-ingress):
  curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable traefik" sh -

• Specify a K3s version:
  curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.28.5+k3s1" sh -

• Bind to a specific network interface:
  curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--bind-address 192.168.1.100 --advertise-address 192.168.1.100" sh -

• Use an external database (PostgreSQL/MySQL) for HA setups:
  curl -sfL https://get.k3s.io | INSTALL_K3s_EXEC="--datastore-endpoint='postgres://user:pass@host:5432/k3s'" sh -

Resource-Constrained Flags

For Raspberry Pi users operating on the edge of their RAM capacity, the following flags are recommended to reduce overhead:

• --disable servicelb: Disables the built-in load balancer.
• --disable traefik: Disables the built-in ingress controller.
• --disable local-storage: Disables the local path provisioner.
• --kubelet-arg="max-pods=50": Limits the number of pods per node to prevent memory exhaustion.

Expanding the Cluster: Adding Worker Nodes

A single-node cluster is a starting point, but distributing workloads across multiple nodes is where the power of Kubernetes is realized.

Retrieving the Join Token

Worker nodes require a security token from the master node to join the cluster. This token ensures that only authorized devices can connect.

• Run the following on the master node:
  sudo cat /var/lib/rancher/k3s/server/node-token

Copy this token for use during the worker installation process.

Joining Workers to the Cluster

SSH into the worker node and set the required environment variables. Replace the placeholders with the actual master IP and the copied token.

• Define variables:
  K3S_MASTER_IP="192.168.1.100"
K3S_TOKEN="K10xxxxxxxxxxxxxxxxxxxxxxxxxxxxx::server:xxxxxxxxxxxxx"

• Install K3s as an agent:
  curl -sfL https://get.k3s.io | K3S_URL=https://$K3S_MASTER_IP:6443 K3S_TOKEN=$K3S_TOKEN sh -

After this process, returning to the master node and running kubectl get nodes will display the worker nodes in the Ready state.

Troubleshooting Common K3s Issues

Despite the simplicity of K3s, specific environmental factors on the Raspberry Pi can lead to failures.

Analysis of K3s on Raspberry Pi Deployment

The deployment of K3s on Raspberry Pi represents a strategic balance between resource limitations and the requirements of a modern orchestration layer. The most critical failure point in this architecture is not the software, but the hardware's physical limitations. Power instability, as noted in the requirement for official PSUs, can lead to random crashes that are often misdiagnosed as software bugs. Similarly, the reliance on microSD cards introduces a significant I/O bottleneck; the recommendation for A2-rated cards or SSDs is an essential consideration for any user moving beyond a basic "hello world" deployment.

From a configuration perspective, the necessity of enabling cgroups via cmdline.txt highlights the gap between a general-purpose Linux OS and a container-optimized environment. This step is non-negotiable, as Kubernetes relies on cgroups for the isolation and limiting of process resources.

The flexibility provided by K3s's installation flags allows for a highly modular approach. By disabling Traefik or ServiceLB, users can reclaim valuable RAM, which is the primary constraint on the Raspberry Pi 4 and 5. This modularity ensures that K3s can scale from a single-node learning environment to a multi-node edge cluster without requiring a complete overhaul of the installation logic. Ultimately, the use of K3s on Raspberry Pi democratizes high-level infrastructure management, providing a low-cost gateway into the world of DevOps and cloud-native architecture.

Sources

1. OneUptime
2. PicoCluster
3. KevsRobots