K3s Deployment on Debian Architecture

The implementation of K3s on Debian Linux represents a strategic shift toward lightweight, certified Kubernetes distributions optimized for resource-constrained environments. K3s is not a fork of Kubernetes; rather, it is a distribution that intentionally maintains a high degree of fidelity to the upstream Kubernetes project. While it maintains a small set of patches—typically well under 1000 lines—to facilitate its specific deployment model, its primary goal is to avoid divergence from core Kubernetes functionality. By packaging additional components and services, K3s provides an opinionated selection of technologies for ingress, storage classes, network policies, service load balancers, and container runtimes, thereby eliminating the manual overhead typically associated with vanilla Kubernetes installations.

The naming convention of K3s is a mathematical and visual play on the term Kubernetes. Since Kubernetes is a ten-letter word stylized as k8s, K3s is conceived as something half as big, with the digit 3 visually representing an 8 cut in half vertically. This distribution is specifically engineered for IoT, Edge computing, and CI environments, where the overhead of a full Kubernetes installation would be prohibitive. It is packaged as a single binary under 70MB, which drastically reduces the dependencies and steps required to install, run, and auto-update production clusters.

System Hardware and Architecture Requirements

Before initiating the installation process on a Debian system, it is critical to evaluate the hardware and architectural compatibility to ensure cluster stability. K3s is designed to be highly portable, supporting a wide range of processor architectures.

The supported architectures include:

• x86_64
• armhf
• arm64/aarch64

The ability to run on ARM64 and ARMv7 enables the deployment of K3s on hardware as small as a Raspberry Pi or as powerful as an AWS a1.4xlarge server with 32GiB of RAM. This flexibility allows developers to maintain parity between their edge devices and their cloud infrastructure.

In terms of system resources, while K3s is lightweight, it does have baseline requirements that must be met by each node. These requirements cover the K3s binary and its packaged components, excluding the resources that will be consumed by the actual workloads (pods) deployed to the cluster.

The recommended minimum resources for a controller server are:

• RAM: 1GB (minimum)
• vCPUs: 2

For those implementing a high-availability (HA) setup, the number of controller servers is pivotal. Deploying only two controller servers can lead to resource contention, characterized by 100% CPU utilization as the nodes "fight" for control. A minimum of three controller servers is recommended. This configuration ensures a consensus mechanism where two servers can override a third, maintaining cluster stability during a node failure.

Debian Operating System Prerequisites

Deploying K3s on Debian requires specific OS-level configurations to prevent networking conflicts and ensure the container runtime can function correctly.

One of the primary prerequisites is the uniqueness of hostnames. Two nodes within the same cluster cannot share the same hostname. If an automated provisioning system is used that might reuse hostnames, users must implement one of the following strategies:

• Use the --with-node-id option to append a random suffix to each node.
• Use the --node-name flag or the $K3S_NODE_NAME environment variable to assign a unique identity to each node.

For Debian systems, specifically those utilizing older releases, the nm-cloud-setup service and timer must be disabled. Failure to do so may interfere with the network configuration of the cluster. The following commands are used to disable these services:

systemctl disable nm-cloud-setup.service nm-cloud-setup.timer

Following the disablement of these services, a system reboot is required to ensure the changes take effect:

reboot

Additionally, users of older Debian releases should be aware of known iptables bugs that may affect cluster communication.

Network Configuration and Firewall Management

The network layer is the most critical component of a K3s deployment. Because K3s leverages specific ports for the API server, pod communication, and service discovery, the firewall must be configured precisely.

It is generally recommended to turn off the Uncomplicated Firewall (UFW) or firewalld entirely to avoid configuration errors:

ufw disable
systemctl disable firewalld --now

However, if a security policy requires the firewall to remain active, specific rules must be implemented. For UFW users, the following rules are mandatory:

• Allow API server traffic: ufw allow 6443/tcp
• Allow pod traffic: ufw allow from 10.42.0.0/16 to any
• Allow service traffic: ufw allow from 10.43.0.0/16 to any

For those using firewalld, the corresponding rules are:

• API server: firewall-cmd --permanent --add-port=6443/tcp
• Pods: firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16
• Services: firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16
• Apply changes: firewall-cmd --reload

Crucially, if the default CIDR for pods or services is modified, these firewall rules must be updated accordingly to prevent the blocking of internal cluster traffic.

The K3s architecture utilizes reverse tunneling, meaning nodes make outbound connections to the server. Consequently, nodes do not need to listen on most ports. However, certain dependencies require specific port accessibility:

• Metrics Server: All nodes must be accessible to each other on port 10250.
• High Availability (embedded etcd): Server nodes must be accessible to each other on ports 2379 and 2380.
• VXLAN: Port 8472 is used for VXLAN. This port should never be exposed to the public internet, as it would allow external access to the cluster network.

The Flannel CNI plugin, used by K3s, relies on the Bridge CNI plugin to create a Layer 2 (L2) network. This architecture introduces a risk where rogue pods with NET_RAW capabilities can perform ARP spoofing attacks.

K3s Installation Process on Debian

The installation of K3s on Debian can be achieved through several methods, depending on whether the user requires a simple single-node setup or a complex high-availability cluster with an external database.

For a basic, rapid installation, the standard curl command is used:

curl -sfL https://get.k3s.io | sh -

This process installs the K3s binary and configures it as a systemd service. To verify the installation and ensure the node is in a "Ready" state, the following command is executed:

sudo k3s kubectl get node

In scenarios where the user prefers to avoid the default Traefik ingress controller, the installation command can be modified:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable traefik" sh -s -

For local development on Debian, users may further enhance their setup by enabling IP forwarding:

sudo sysctl -w net.ipv4.ip_forward=1

This configuration allows the host to route traffic between different network interfaces, which is often necessary when using custom container registries (such as Docker) on the same host.

Advanced Server Configuration and High Availability

For production environments, K3s can be configured as a master controller server integrated with an external database. This removes the dependency on embedded etcd and allows for more robust scaling.

To connect K3s to an external MySQL database, the following environment variables must be defined and exported:

DB_USER="kubernetes"
DB_PASSWORD="mySuperSecretPassword"
DB_IP="192.168.0.72"
DB_PORT=3306
DB_NAME="kubernetes"
export K3S_DATASTORE_ENDPOINT="mysql://$DB_USER:$DB_PASSWORD@tcp($DB_IP:$DB_PORT)/$DB_NAME"

Once the datastore endpoint is exported, the server is installed using a command that specifies the server role and the Load Balancer IP for TLS SAN (Subject Alternative Name) support. This ensures that the API server is accessible via the Load Balancer's IP.

LOAD_BALANCER_IP=192.168.0.73
sudo curl -sfL https://get.k3s.io | sh -s - server --node-taint CriticalAddonsOnly=true:NoExecute --tls-san $LOAD_BALANCER_IP

The use of --node-taint CriticalAddonsOnly=true:NoExecute ensures that only critical system pods are scheduled on the controller nodes, preventing application workloads from consuming resources needed for cluster management.

Agent Node Integration

Once the server (master) is operational, additional nodes can be added to the cluster as agents. Agent nodes handle the actual workloads and do not run the control plane.

To join an agent to the cluster, the agent must be provided with the server's URL and a secret node token. The token is retrieved from the server node at the following location:

/var/lib/rancher/k3s/server/node-token

The agent is then started using the following command:

sudo k3s agent --server https://myserver:6443 --token ${NODE_TOKEN}

This establishes the connection to the master server, allowing the agent to receive instructions and deploy pods as directed by the Kubernetes scheduler.

Special Considerations for Raspberry Pi (Debian-Based)

Because Raspberry Pi OS is based on Debian, it inherits many of the same requirements, but it also introduces specific hardware-level configuration needs, particularly regarding cgroups.

Cgroups (control groups) are essential for the systemd service to start K3s. In standard Raspberry Pi OS installations, cgroups are not enabled by default. To enable them, users must append specific parameters to the kernel command line.

The location of the configuration file depends on the OS version:

• Raspberry Pi OS (Current): /boot/firmware/cmdline.txt
• Debian 11 and older Pi OS releases: /boot/cmdline.txt

The following strings must be appended to the line in cmdline.txt:

cgroup_memory=1 cgroup_enable=memory

An example of a complete cmdline.txt would look like this:

console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory

Furthermore, users operating Ubuntu on Raspberry Pi (versions 21.10 through 23.10) must be aware that vxlan support was moved into a separate kernel module, which may require additional loading to ensure network connectivity.

Component Comparison and Specifications

The following table summarizes the key architectural specifications and requirements for K3s on Debian systems.

Detailed Analysis of the K3s Distribution Model

The distinction between K3s and vanilla Kubernetes is fundamental to understanding its utility on Debian. Vanilla Kubernetes requires the manual selection and configuration of a Container Runtime Interface (CRI), a Container Network Interface (CNI), and various other components. K3s streamlines this by making opinionated choices.

By integrating these components into a single binary, K3s reduces the surface area for installation errors. For instance, the inclusion of a default service load balancer and ingress controller allows a user to deploy a functional cluster in seconds, rather than hours. This distribution model is what enables K3s to operate effectively in "unattended" locations. In a remote edge environment, the ability for the system to auto-update and run with minimal dependencies is the difference between a sustainable deployment and a maintenance nightmare.

The commitment to remaining close to upstream Kubernetes ensures that any YAML manifest written for a standard Kubernetes cluster will work on K3s. This prevents vendor lock-in and allows organizations to scale their workloads from a small K3s cluster on Debian servers to a massive EKS or GKE cluster in the cloud without rewriting their deployment logic.

The resource efficiency is achieved not by removing Kubernetes features, but by optimizing how they are delivered. By eliminating the need for several external dependencies and packaging the core logic into a single binary, K3s minimizes the memory footprint and CPU cycles required to maintain the control plane. This makes it the ideal choice for the "Edge" where hardware is often limited to a few gigabytes of RAM.

Sources

1. willhaley.com
2. blog.programster.org
3. docs.k3s.io
4. k3s-io.github.io
5. github.com/k3s-io/k3s