K3s Ubuntu Cluster Provisioning

The architectural landscape of container orchestration has shifted toward specialized distributions that prioritize efficiency over monolithic feature sets. K3s represents a pivotal shift in this evolution, serving as a lightweight Kubernetes distribution specifically engineered for resource-constrained environments, edge computing, and rapid developer iteration. Developed by Rancher Labs, K3s streamlines the traditional Kubernetes experience by stripping away unnecessary cloud provider integrations and legacy features. This refinement allows the distribution to be bundled into a single binary weighing less than 100 MB. A critical architectural change is the default replacement of the heavy etcd database with SQLite, which significantly reduces the overhead required to maintain the cluster state.

The real-world consequence of this streamlined design is a dramatic reduction in startup time and hardware requirements. A K3s cluster can transition from a powered-off state to a fully operational environment in under 30 seconds. Furthermore, it allows Kubernetes to run on hardware with as little as 512 MB of RAM, expanding the reach of container orchestration to ARM64 devices and small-form-factor hardware. This makes K3s an essential tool for those who require the standard Kubernetes API without the infrastructure overhead associated with full-scale production clusters.

Architectural Advantages and Deployment Scenarios

The selection of K3s over a standard Kubernetes distribution is driven by specific operational requirements where friction must be minimized. Because it is delivered as a single binary, it removes the need to pre-install a container runtime, which is a common pain point in standard Kubernetes deployments.

The following table outlines the specific scenarios where K3s provides a distinct advantage over full Kubernetes:

Scenario K3s Advantage
Home labs and CI runners Single binary, no container runtime to pre-install
Edge devices (Raspberry Pi, NUCs) Low memory footprint, ARM64 support
Development environments Fast spin-up, easy teardown
Air-gapped installations Offline install script available
Learning Kubernetes Same API, less infrastructure overhead

For users operating in home labs or CI (Continuous Integration) runners, the lack of a pre-installation requirement for the container runtime allows for nearly instantaneous environment provisioning. For those deploying to edge devices like Raspberry Pis or Intel NUCs, the ARM64 support and low memory footprint ensure that the orchestration layer does not consume the resources intended for the actual workloads. In development environments, the ease of spin-up and teardown allows engineers to test manifests in a production-like environment without dedicating significant hardware. Even in air-gapped environments where internet connectivity is restricted, K3s provides an offline installation script to ensure continuity.

However, it is important to note the boundaries of this distribution. If a project requires multi-master High Availability (HA) utilizing an external etcd cluster and every single upstream feature available in the Kubernetes ecosystem, standard kubeadm or a managed cloud service would be a more appropriate choice.

System Requirements and Environmental Preparation

Before initiating the installation process on Ubuntu, the underlying system must meet specific hardware and software benchmarks to ensure stability. The distribution is compatible with Ubuntu 20.04 LTS, 22.04 LTS, and 24.04 LTS.

The hardware requirements are scaled based on the intended use:

  • Minimum Requirements: 1 CPU core and 512 MB RAM.
  • Recommended Requirements: 2 CPU cores and 2 GB RAM for real-world workloads.
  • Administrative Access: Root or sudo privileges are mandatory.

The impact of these requirements is that K3s can be deployed on almost any modern virtual machine or physical device. To verify the current operating system version and ensure compatibility, the following command should be executed:

lsb_release -a

Once the version is confirmed, the system packages must be updated to ensure that the latest security patches and dependencies are in place. This prevents compatibility conflicts during the script execution.

sudo apt update && sudo apt upgrade -y

In environments where a firewall is active, specific ports must be opened to allow the Kubernetes components to communicate. Failure to open these ports will result in a non-functional cluster where nodes cannot join the server or the API server remains unreachable.

  • Port 6443: Used for the Kubernetes API.
  • Port 8472/UDP: Used for Flannel VXLAN networking.
  • Port 10250: Used for the kubelet.

K3s Server Node Installation Process

The installation of the K3s server node is handled through a streamlined, one-liner installation script. This script is designed to handle the complexities of binary downloading, service configuration, and system integration.

To initiate the installation of the K3s cluster, the following command is executed:

curl -sfL https://get.k3s.io | sh -

This execution triggers a series of automated steps that transform a standard Ubuntu machine into a Kubernetes server. The internal mechanics of the script involve the following actions:

  • Downloading the K3s binary to /usr/local/bin/k3s.
  • Creating a systemd service file at /etc/systemd/system/k3s.service.
  • Generating the necessary kubeconfig file at /etc/rancher/k3s/k3s.yaml.
  • Starting the K3s service immediately upon completion.

The script also handles the creation of essential symlinks to ensure that common Kubernetes tools are accessible globally. These include symlinks for kubectl, crictl, and ctr located in /usr/local/bin/. Additionally, it generates a killall script at /usr/local/bin/k3s-killall.sh and an uninstall script at /usr/local/bin/k3s-uninstall.sh.

The output of the installation process provides a transparent log of these actions. For example, the log will indicate the release channel being used (stable), the specific version being installed (such as v1.27.7+k3s2), and the verification of the binary download.

Once the installation is finished, the status of the service should be verified to ensure the cluster is active.

sudo systemctl status k3s

A successful installation will display an "active (running)" status. At this point, users can execute kubectl commands directly to verify the health of the cluster.

kubectl get pods -n kube-system

This command allows the user to visualize all the required pods and components that comprise a production-grade Kubernetes cluster.

Configuring kubectl Access and Permissions

K3s bundles kubectl within its binary, meaning no separate installation of the command-line tool is required. However, a critical security and configuration detail is that the kubeconfig file is stored at /etc/rancher/k3s/k3s.yaml.

By default, this file is only accessible to the root user. This means that any kubectl command executed by a non-root user will fail due to lack of permissions to read the configuration. This requires users to either execute commands with sudo or modify the permissions of the kubeconfig file to allow their specific user account to access the cluster.

K3s Agent Node Installation and Connectivity

To expand a single-node cluster into a multi-node cluster, agent nodes must be deployed. An agent node functions as a worker that can run pods but does not manage the cluster state.

To connect an agent node to the server, two specific pieces of information are required:

  • serverIP: The IP address of the server node.
  • node-token: A unique security token generated by the server node during installation.

The server IP allows the agent to locate the master node on the network, while the node-token ensures that only authorized nodes can join the cluster. This prevents unauthorized machines from injecting themselves into the orchestration environment.

Customizing K3s Configuration

K3s allows for modification of the cluster configuration both during the initial installation and after the cluster is already running. A common example of this is the removal of Traefik, the default ingress controller.

Installation-Time Configuration

To disable Traefik during the initial installation, an environment variable can be passed to the installation script. This prevents the Traefik resources from ever being created in the kube-system namespace.

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=traefik" sh -

After running this, the user can verify that no Traefik-related objects exist by listing all resources in the system namespace:

sudo kubectl get all -n kube-system

Post-Installation Configuration

If K3s is already installed, configuration changes can be made via the K3s configuration file located at /etc/rancher/k3s/config.yaml. This file allows for a declarative approach to cluster management.

To disable Traefik post-installation, the user should open the configuration file:

sudo nano /etc/rancher/k3s/config.yaml

The following line should be added to the file:

disable: traefik

After saving the file and exiting the editor, the K3s service must be restarted to apply the new configuration:

sudo systemctl restart k3s

The impact of this change is that the K3s service will delete all resources related to the Traefik installation. This can be verified again using the following command:

sudo kubectl get all -n kube-system

For a complete list of available configuration flags and options, users are encouraged to reference the official documentation at https://docs.k3s.io/cli/server.

Cluster Decommissioning and Uninstallation

When a cluster is no longer needed, or when a user wishes to perform a clean wipe of the virtual machine, K3s provides an automated cleanup mechanism. Manual deletion of files is discouraged as it may leave orphaned network interfaces or systemd services.

The installation process automatically generates a shell script located at /usr/local/bin/k3s-uninstall.sh. This script is designed to perform a full cleanup of the environment.

To execute the uninstallation, the following command is used:

/usr/local/bin/k3s-uninstall.sh

The uninstallation script removes all K3s configurations, binaries, and cluster tools that were created during the installation process. This ensures that the VM is returned to its original state.

To verify that the uninstallation was successful, the user should check the status of the k3s service:

systemctl status k3s

A successful removal will result in an output stating that the k3s.service could not be found.

Analysis of K3s Operational Efficiency

The operational efficiency of K3s on Ubuntu is a direct result of its philosophy of "minimalism without loss of functionality." By replacing etcd with SQLite, K3s eliminates the most resource-intensive part of the Kubernetes control plane. In a standard Kubernetes deployment, etcd requires significant CPU and I/O overhead to maintain consistency across multiple nodes. For small-scale deployments, this is an unnecessary tax on resources.

The move to a single binary also simplifies the lifecycle management of the cluster. In traditional Kubernetes, updating the cluster often involves updating multiple components (kubelet, kube-proxy, API server) individually. K3s collapses these into a single executable, making updates and version migrations significantly more reliable.

Furthermore, the integration with systemd ensures that K3s behaves like a standard Linux service. This allows administrators to use familiar tools like systemctl for monitoring, starting, and stopping the cluster. The impact is a lowered barrier to entry for Linux administrators who may not be experts in Kubernetes but are comfortable with Ubuntu system administration.

The ability to disable components like Traefik via the config.yaml file provides a level of flexibility that allows users to build a custom stack. For instance, a user might prefer to use NGINX or HAProxy as an ingress controller. By disabling the default Traefik installation, the user avoids resource contention and configuration conflicts, allowing for a leaner, more tailored architecture.

In summary, the deployment of K3s on Ubuntu represents a strategic optimization for those who need the power of the Kubernetes API without the burden of its original infrastructure requirements. Whether deployed on an AWS EC2 instance running Ubuntu 24.04 or a Raspberry Pi in a home lab, K3s provides a stable, fast, and resource-efficient foundation for modern containerized workloads.

Sources

  1. Devtron
  2. OneUptime
  3. DigitalOcean

Related Posts