Kubernetes 1.23 Architectural Evolution and Deployment Framework

The release of Kubernetes 1.23 represents a pivotal moment in the maturation of the container orchestration ecosystem, introducing 45 distinct enhancements. To understand the scale of this release, it must be viewed in the context of its predecessors; while Kubernetes 1.22 introduced 56 enhancements and 1.21 brought 50, version 1.23 focuses on the refinement of existing capabilities and the graduation of critical features to a stable state. Of these 45 enhancements, 11 have officially graduated to Stable (GA), 15 are iterative improvements to existing features, and 19 are entirely new introductions. This release is characterized by a shift toward modularity, specifically the decoupling of the Container Storage Interface (CSI) from the Kubernetes core, and a significant push toward increasing the accessibility of the platform for non-technical users through the implementation of OpenAPI v3.

Core Feature Graduations and General Availability

The most significant impact of Kubernetes 1.23 is the graduation of several high-impact features to General Availability (GA). GA status indicates that these features have reached a level of stability and reliability suitable for production environments, removing the "alpha" or "beta" caveats that previously limited their deployment in mission-critical systems.

• CronJobs: This feature has reached GA, providing a stable mechanism for executing jobs on a time-based schedule.
• IPv4/IPv6 Dual-Stack Support: Now GA, this allows clusters to handle both IPv4 and IPv6 traffic simultaneously, expanding networking flexibility.
• Ephemeral Volumes: These have graduated to GA, allowing for the use of temporary storage that exists only for the duration of the pod's lifecycle.
• HPA API: The Horizontal Pod Autoscaler API has graduated to GA, ensuring a stable interface for scaling pods based on observed CPU utilization or other custom metrics.

Advanced Node Management and Resource Optimization

Kubernetes 1.23 introduces sophisticated controls for how physical hardware is utilized by containers, aiming to eliminate the "noisy neighbor" effect where multiple containers competing for the same physical core degrade each other's performance.

CPU Manager Policy Enhancements

The CPU Manager Policy Options have graduated to beta in version 1.23. In previous versions, the default CPU allocation strategy often allocated containers as individual virtual cores. This approach frequently led to multiple containers sharing the same physical core, resulting in unpredictable performance fluctuations.

The introduction of the full-pcpus-only policy changes this dynamic. This policy guarantees that pods will only be started if their CPU requests can be fulfilled using full physical cores. For the end user, this means deterministic performance and the elimination of resource contention at the physical hardware level, which is critical for high-performance computing (HPC) workloads.

Container and Pod Statistics Transition

A major architectural shift is occurring in how Kubernetes handles telemetry and resource monitoring. Enhancement #2371 focuses on "cAdvisor-less, CRI-full Container and Pod stats," which has graduated to Alpha.

Historically, the summary API and the /metrics/cadvisor endpoint relied heavily on cAdvisor to gather metrics, with the Container Runtime Interface (CRI) providing supplementary data. The new approach seeks to retrieve all necessary statistics directly from the CRI. By removing the dependency on cAdvisor, Kubernetes reduces the overhead of the Kubelet and streamlines the pipeline for monitoring container resource usage.

Networking and Service Connectivity

Networking in Kubernetes 1.23 is defined by a move toward topology awareness and the adoption of modern protocols to improve service resilience.

Topology Aware Hints

Enhancement #2333, Topology Aware Hints, has graduated to Beta. This feature is designed to replace the ServiceTopology feature gate that was introduced back in Kubernetes 1.17. Topology Aware Hints allow the network to route traffic to the closest available endpoint based on the topology of the cluster. This reduces network latency and decreases the cost of data transfer between availability zones.

gRPC Probe Integration

The addition of gRPC probes (#2727) provides a native way for Kubernetes to monitor the health of services using the gRPC protocol. Traditionally, SREs relied on HTTP probes to check if a service was alive. Given the growth of gRPC in cloud-native applications, this native support allows developers to implement more precise health checks. This results in higher service availability, as the orchestrator can detect failures in gRPC-based services more accurately and trigger restarts or routing changes faster.

Storage Architecture and Modularity

One of the most strategic efforts in Kubernetes 1.23 is the movement of CSI (Container Storage Interface) drivers out of the Kubernetes core. This is exemplified by enhancement #2317, which delegates the FSGroup to the CSI Driver instead of the Kubelet.

By shifting this responsibility, Kubernetes achieves real modularity. The Kubelet no longer needs to handle the specifics of volume ownership and permissions; instead, the CSI driver—which is specific to the storage vendor—handles these operations. This reduces the complexity of the Kubernetes core code and allows storage vendors to implement FSGroup logic that is optimized for their specific hardware.

Other storage-related enhancements include:

• Skip Volume Ownership Change: Allows for faster pod startup by skipping unnecessary ownership changes on volumes.
• Config FSGroup Policy in CSI Driver object: Provides more granular control over how the FSGroup policy is applied within the CSI driver.
• Generic Ephemeral Inline Volumes: Extends the capability of using ephemeral storage within pod specifications.

Windows Container Support and Privileged Mode

Kubernetes 1.23 expands the capabilities of Windows containers to match the flexibility found in Linux containers. Specifically, Windows containers can now run in privileged mode.

This is achieved by setting the windowsOptions.hostProcess flag in the Pod specification. In this mode, the pod runs directly on the host, and all containers within that pod must be configured as HostProcess containers. While this is not recommended for general application workloads due to the security risks associated with host access, it is essential for:

• Security tools that require deep visibility into the host operating system.
• Monitoring agents that must collect system-level metrics from the Windows host.
• Administrative utilities that perform low-level OS configuration.

API Evolution and OpenAPI v3

The integration of OpenAPI v3 is a transformative update for the Kubernetes API. This update aims for full transparency and introduces support for fields that were discarded in OpenAPI v2, such as:

• default
• nullable
• oneOf
• anyOf

Furthermore, OpenAPI v3 implements a separate specification published per Kubernetes group version, accessible at the $cluster/openapi/v3/apis/<group>/<version> endpoint. This structure improves performance and discovery. For users, the most visible impact is the ability for User Interfaces (UIs) to display dropdowns for Enum types with proper validation. This reduces the barrier to entry for less technical users, as the API becomes self-documenting and easier to interact with via GUI tools.

Deployment and Cluster Initialization

For users deploying Kubernetes 1.23 using kubeadm and containerd, the initialization process requires specific configuration to ensure the control plane and nodes are synchronized.

Image Acquisition

Before initializing the cluster, the required images for version 1.23 must be pulled from the registry using the following command:

bash sudo kubeadm config images pull --cri-socket /run/containerd/containerd.sock --kubernetes-version v1.23.0

Control Plane Initialization

The initialization of the control plane node involves configuring the pod network CIDR (e.g., for Flannel) and defining the control plane endpoint. The use of a public IP for the endpoint is necessary if the cluster is to be accessed via Kubeconfig from outside the node.

The execution command is as follows:

bash sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --upload-certs --kubernetes-version=v1.23.0 --control-plane-endpoint=212.2.245.195 --ignore-preflight-errors=Mem --cri-socket /run/containerd/containerd.sock

During this process, the system generates critical certificates in /etc/kubernetes/pki, including the CA, apiserver, and other component certificates. The process concludes with the creation of a kubelet-config-1.23 ConfigMap in the kube-system namespace. Note that the naming convention kubelet-config-1.23 is currently deprecated and will eventually transition to kubelet-config once the UnversionedKubeletConfigMap feature gate graduates to Beta.

Deprecations and Removals

Kubernetes 1.23 introduces several removals and deprecations that require immediate attention from cluster administrators to avoid breaking changes in future updates.

API and Metric Removals

The following changes have been implemented to streamline the API and monitoring:

• Metrics: The apiserver_longrunning_requests metric has been introduced to replace the apiserver_longrunning_gauge metric.
• Controller-manager: The --port and --address flags have been rendered ineffective and are scheduled for complete removal in version 1.24.
• Kube-scheduler: The metricsBindAddress and healthzBindAddress fields within kubescheduler.config.k8s.io/v1beta1 are now expected to be empty.
• Authentication/Authorization: The --authorization-kubeconfig and --authentication-kubeconfig flags must be correctly configured to ensure the authentication and authorization systems function.

Comparative Analysis of Feature Stages

The distribution of features in Kubernetes 1.23 reveals a strategic focus on stabilizing the core while experimenting with node-level efficiencies.

Analysis of Architectural Impact

Kubernetes 1.23 is not a release of "massive" singular changes, but rather a collection of "transcendent details" that collectively improve the operational stability of the platform. The shift toward OpenAPI v3 and gRPC probes demonstrates a commitment to the developer experience (DX), making the platform more accessible and its services more resilient.

From an infrastructure perspective, the delegation of FSGroup to CSI drivers and the transition to CRI-based statistics signal a broader architectural goal: the "thinning" of the Kubelet. By removing vendor-specific logic and third-party dependencies like cAdvisor from the core node agent, Kubernetes becomes more maintainable and less prone to regressions when storage or runtime providers update their implementation.

The introduction of the full-pcpus-only CPU policy is particularly critical for the "noisy neighbor" problem. In high-density clusters, the ability to guarantee physical core allocation transforms Kubernetes from a general-purpose orchestrator into a viable platform for latency-sensitive and high-throughput applications.

In summary, Kubernetes 1.23 balances the necessity of cleaning up technical debt (through deprecations and removals) with the strategic expansion of its capabilities (through GA graduations and the introduction of modern protocols). The result is a version that is more modular, more performant at the hardware level, and more welcoming to a broader range of technical users.

Sources

1. Sysdig
2. Amazon AWS
3. KubeSimplify
4. Kubernetes Blog