SQLite-Powered K3s Edge Architecture and Control Plane Migration

The deployment of Kubernetes at the edge represents a fundamental shift in infrastructure design, moving away from centralized cloud clusters toward distributed, localized compute. For many edge scenarios—such as IoT gateways, remote monitoring stations, and small retail kiosks—the overhead of a full Kubernetes distribution is prohibitive. K3s addresses this by packaging core components into a single binary, and when paired with SQLite, it transforms into a lean, high-performance engine capable of running on extremely modest hardware. Unlike standard Kubernetes, which relies on etcd for its distributed key-value store, K3s uses SQLite by default for single-server deployments. This architectural choice removes the need for external database management, significantly reducing the memory footprint and eliminating network dependencies for maintaining the cluster state.

The decision to utilize SQLite is a strategic tradeoff. While etcd is designed for high availability across multiple control plane nodes, SQLite is optimized for simplicity and resource efficiency on a single node. This makes it the ideal choice for environments where high availability is less critical than low resource consumption and ease of maintenance. By leveraging a single-file database, administrators can manage the entire state of their Kubernetes cluster with simple file-system operations, simplifying everything from initial deployment to disaster recovery.

Resource Requirements and Hardware Compatibility

Deploying K3s with SQLite allows for the utilization of low-power hardware that would otherwise be incapable of running a container orchestrator. The resource requirements are remarkably low, enabling the use of ARM-based single-board computers and legacy x86 hardware.

The minimum hardware specifications for a functional K3s SQLite instance are as follows:

Resource Minimum Requirement Recommended Specification
RAM 512MB 1GB
CPU 1 Core 2 Cores
Disk Space 2GB (K3s system only) 2GB + workload storage
Architecture ARM32v7, ARM64, x86_64 ARM64 or x86_64

The impact of these low requirements is significant for edge deployments. For instance, using a Raspberry Pi 4 with 4GB of RAM provides a comfortable operating environment, leaving ample overhead for actual containerized workloads. The architecture support ensures that whether an organization is using industrial ARM-based gateways or standard Intel/AMD NUCs, K3s remains a viable option.

Installation and Initial Configuration

Installing K3s on a single-node edge device is streamlined through a shell script that automates the binary deployment and service configuration. Because SQLite is the default datastore for single-server installations, no additional database configuration is required during the initial phase.

For a standard installation, the following command is used:

curl -sfL https://get.k3s.io | sh -

However, in resource-constrained environments, it is critical to disable unnecessary components to save memory and CPU cycles. The following command implements a tuned installation:

curl -sfL https://get.k3s.io | sh -s - \ --write-kubeconfig-mode 644 \ --disable servicelb \ --disable traefik \ --kube-apiserver-arg='max-requests-inflight=100' \ --kube-apiserver-arg='max-mutating-requests-inflight=50'

The specific flags used in this command serve several purposes:

  • --write-kubeconfig-mode 644: This ensures the kubeconfig file is readable by non-root users, facilitating easier management.
  • --disable servicelb: This removes the default Service LoadBalancer, which is often redundant in a single-node edge setup.
  • --disable traefik: This removes the default Traefik Ingress controller, reducing the memory overhead significantly if a different ingress solution is used or if the edge device does not require external HTTP access.
  • --kube-apiserver-arg='max-requests-inflight=100': This limits the number of active requests to the API server, preventing the system from becoming overwhelmed during spikes in activity.
  • --kube-apiserver-arg='max-mutating-requests-inflight=50': This specifically limits requests that change the state of the cluster, protecting the SQLite database from excessive write contention.

Once installed, administrators can verify the status of the cluster using:

kubectl get nodes
k3s check-config

To confirm that the cluster is indeed utilizing the SQLite datastore, one can inspect the state database file located at:

sudo cat /var/lib/rancher/k3s/server/db/state.db

Aggressive Resource Optimization for Ultra-Low Memory

For devices operating with 1GB of RAM or less, a basic installation may still be too heavy. In these cases, a custom configuration file is required to strip K3s down to its absolute essentials.

First, the configuration directory must be created:

sudo mkdir -p /etc/rancher/k3s

Then, a config.yaml file is created with the following settings:

yaml write-kubeconfig-mode: "0644" disable: - servicelb - traefik - local-storage - metrics-server kube-apiserver-arg: - "max-requests-inflight=50" - "max-mutating-requests-inflight=25"

The removal of local-storage and metrics-server further minimizes the CPU and memory footprint. The metrics-server is particularly resource-intensive as it constantly polls for resource usage data. By disabling these, the system can prioritize the stability of the primary workloads over monitoring and local volume management.

Storage Management at the Edge

While K3s handles the orchestration, managing persistent data on a single node requires a focused approach. In an edge environment, using a PersistentVolumeClaim (PVC) allows workloads to request storage from the host.

An example of a PVC for sensor data would look like this:

yaml PersistentVolumeClaim metadata: name: sensor-data spec: accessModes: - ReadWriteOnce storageClassName: local-path resources: requests: storage: 1Gi

In this configuration, the local-path storage class is used to map the claim directly to a directory on the node's disk. This is the most efficient way to handle persistence in a single-node SQLite cluster, as it avoids the complexity of distributed storage layers.

The SQLite Datastore: Backup and Recovery

Because the entire state of a single-node K3s cluster resides in a single SQLite file, the backup process is significantly simpler than in etcd-based clusters. However, the SQLite file alone is not enough; the server token must also be backed up to ensure the restored cluster can decrypt its own data.

Automated Backup Implementation

To prevent data loss, a bash script can be implemented to automate the backup of the database and the configuration files.

sudo tee /usr/local/bin/k3s-backup.sh > /dev/null <<'EOF'
```bash

!/bin/bash

BACKUPDIR="/var/backups/k3s"
DATE=$(date +%Y%m%d-%H%M%S)
mkdir -p $BACKUP
DIR

Stop K3s briefly for consistent backup

systemctl stop k3s

Backup SQLite database

cp /var/lib/rancher/k3s/server/db/state.db $BACKUP_DIR/state-$DATE.db

Backup certificates and config

tar -czf $BACKUP_DIR/k3s-config-$DATE.tar.gz \
/etc/rancher/k3s \
/var/lib/rancher/k3s/server/tls \
/var/lib/rancher/k3s/server/token

Start K3s

systemctl start k3s

Cleanup old backups (keep last 7 days)

find $BACKUPDIR -name "*.db" -mtime +7 -delete
find $BACKUP
DIR -name "*.tar.gz" -mtime +7 -delete
echo "Backup completed: $BACKUP_DIR"
EOF
```

After creating the script, it must be made executable:

sudo chmod +x /usr/local/bin/k3s-backup.sh

To ensure this runs daily at 2 AM, it is added to the system crontab:

sudo crontab -e

Add the following line:

0 2 * * * /usr/local/bin/k3s-backup.sh

Restoring from Backup

In the event of database corruption or system failure, the restore process involves stopping the K3s service and replacing the state file.

sudo systemctl stop k3s

To restore the database:

sudo cp /var/backups/k3s/state-20260209-020000.db /var/lib/rancher/k3s/server/db/state.db

To restore the configuration and tokens:

sudo tar-xzf /var/backups/k3s/k3s-config-20260209-020000.tar.gz -C /

Finally, restart the service and verify the cluster health:

sudo systemctl start k3s
kubectl get nodes
kubectl get pods --all-namespaces

Control Plane Promotion and Node Migration

A critical challenge when running K3s on edge hardware like Raspberry Pis is the wear and tear on MicroSD cards. Kubernetes is a write-heavy application, and the frequent updates to the SQLite database can lead to SD card failure within a year. To mitigate this, administrators can "promote" the control plane from an SD-card-backed node to an SSD-backed node.

Since K3s bundles its core components, the only way to perform a control plane promotion with SQLite is through a full backup and restore of the datastore and token. This process involves approximately 15 minutes of downtime.

Migration Step-by-Step

  1. Capture State on Existing Control Plane:
    First, stop the service to ensure data consistency.

sudo systemctl stop k3s

Copy the essential files to a backup directory:

sudo cp /var/lib/rancher/k3s/server/db/state.db ~/k3s_backup/
sudo cp /var/lib/rancher/k3s/server/token ~/k3s_backup/

  1. Wipe Existing Cluster:
    To avoid conflicts, K3s must be removed from all nodes, including workers.

sudo k3s-killall.sh
sudo rm -rf /etc/rancher/k3s /var/lib/rancher/k3s

  1. Prepare the New SSD-Backed Control Plane:
    Transfer the state.db and token files to the new node via SCP. For Raspberry Pi users, specific kernel arguments must be added to /boot/firmware/cmdline.txt to ensure cgroup compatibility, otherwise, the installation will fail.

Add these arguments to the end of the line:

cgroup_enable=memory cgroup_memory=1 systemd.unified_cgroup_hierarchy=1

Reboot the device:

sudo reboot

  1. Restore and Initialize:
    Install K3s using the standard script:

curl -sfL https://get.k3s.io | sh -

Immediately stop the service to perform the restoration:

sudo systemctl stop k3s

sudo cp state.db /var/lib/rancher/k3s/server/db/state.db
sudo cp token /var/lib/rancher/k3s/server/token

Finally, remove conflicting generated data and restart:

sudo rm -rf /var/lib/rancher/k3s/server/tls
sudo rm -f /etc/rancher/k3s/k3s.yaml

By moving the control plane to an SSD, the risk of a catastrophic failure due to disk wear is minimized, and the recovery of a worker node becomes a trivial task compared to recovering the entire cluster state.

Security Hardening for Exposed Edge Nodes

Edge nodes are often physically accessible or located in less secure environments, making security hardening mandatory. This involves restricting network access and disabling unnecessary API features.

Network Firewall Configuration

Use ufw to restrict traffic to only necessary ports:

sudo ufw allow 22/tcp
sudo ufw allow 6443/tcp
sudo ufw enable

Port 6443 is the Kubernetes API port; in a production edge environment, this should be restricted to the management network only.

K3s API Hardening

Further security can be achieved by modifying the config.yaml to disable anonymous authentication and enable audit logging:

sudo tee -a /etc/rancher/k3s/config.yaml > /dev/null <<EOF
yaml disable: - servicelb - traefik kube-apiserver-arg: - "anonymous-auth=false" - "audit-log-path=/var/log/k3s-audit.log" - "audit-log-maxage=30"
EOF

sudo systemctl restart k3s

This configuration ensures that no one can interact with the API without proper credentials and provides an audit trail of all activities, which is essential for compliance in industrial settings.

Maintenance and Disaster Recovery

Maintaining a fleet of edge nodes requires a strategy for upgrades and a method for detecting and fixing database corruption.

Safe Upgrading Procedure

Upgrading K3s on a single-node SQLite cluster should always begin with a backup.

/usr/local/bin/k3s-backup.sh

Once backed up, the new version can be installed over the existing one:

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.28.5+k3s1 sh -

Verify the upgrade with:

k3s --version
kubectl get nodes

Handling SQLite Corruption

In rare cases, the SQLite database may become corrupted due to sudden power loss or hardware failure. To check the integrity of the database, use the sqlite3 command-line tool:

sudo sqlite3 /var/lib/rancher/k3s/server/db/state.db "PRAGMA integrity_check;"

If the check returns anything other than "ok", the database is corrupted. The recovery process is as follows:

sudo systemctl stop k3s
sudo mv /var/lib/rancher/k3s/server/db/state.db /var/lib/rancher/k3s/server/db/state.db.corrupt
sudo cp /var/backups/k3s/state-latest.db /var/lib/rancher/k3s/server/db/state.db
sudo systemctl start k3s

Comparative Analysis of K3s Datastores

Depending on the scale and availability requirements, administrators must choose between SQLite, embedded etcd, or an external database.

Feature SQLite (Embedded) etcd (Embedded) External Database
Setup Complexity Zero (Default) Low Moderate
Memory Overhead 20-30MB 200MB+ Variable (External)
High Availability No (Single Node) Yes (Multi-Node) Yes (DB Managed)
Backup Method File Copy etcd-snapshot DB Dump/Snapshot
Ideal Use Case IoT/Single Node Edge Small-Mid Clusters Enterprise Edge
Token Requirement Mandatory for Restore Mandatory for Restore Managed by DB

The use of the server token file at /var/lib/rancher/k3s/server/token is a universal requirement for K3s. This token is used to encrypt confidential data within the datastore. If a restore is attempted with a different token, the snapshot will be unusable because the encryption keys will not match.

Conclusion

The implementation of K3s with SQLite represents a highly optimized approach to edge computing. By trading off the high availability of etcd for the simplicity of a single-file database, K3s allows Kubernetes to run on hardware as limited as 512MB of RAM and a single CPU core. The operational simplicity is evident in the backup and restore processes, which rely on standard Linux file utilities rather than complex database dump tools.

However, the reliance on SQLite introduces specific vulnerabilities, most notably the susceptibility of SD cards to wear-and-tear in write-heavy Kubernetes environments. The "Control Plane Promotion" strategy—migrating the SQLite state from a volatile SD card to a stable SSD—is a critical maneuver for any administrator looking to maintain long-term stability in an edge cluster. When combined with aggressive resource tuning, strict firewall rules, and automated backup scripts, K3s with SQLite provides a professional-grade orchestration layer that is both resilient and incredibly efficient. The transition from a centralized cloud model to a distributed edge model is made possible by these kinds of lean architectural choices, enabling the deployment of complex containerized applications in the most remote and resource-constrained locations on earth.

Sources

  1. oneuptime.com
  2. danielstanecki.com
  3. docs.k3s.io

Related Posts