Orchestrating GitLab Runner Deployments via Bitnami and Kubernetes Helm Charts

The deployment of a robust version control system and continuous integration/continuous delivery (CI/CD) pipeline is a foundational requirement for any modern software development lifecycle. At the center of this ecosystem is GitLab, a Ruby on Rails web application designed to manage repository permissions, facilitate code reviews, and provide integrated wiki functionality. While GitLab serves as the management plane, the actual execution of jobs—the compiling, testing, and deploying of code—is handled by the GitLab Runner. When deploying these runners within a Kubernetes environment, utilizing a structured approach via Helm charts and optimized images, such as those provided by Bitnami and the official GitLab registries, ensures that the infrastructure is scalable, secure, and maintainable.

The transition from a traditional software project to a managed one begins with the implementation of a Version Control System (VCS). A VCS allows developers to track changes, maintain concurrent versions of a codebase (such as isolating production environments from development environments), and streamline the merging of contributions from multiple contributors. While Git is the primary tool for this distributed version control, GitLab provides the administrative layer necessary for organizational scale. To operationalize this, the GitLab Runner must be configured to interface with the GitLab instance, often leveraging Kubernetes for dynamic scaling of build pods.

Architectural Overview of GitLab Runner in Kubernetes

The GitLab Runner in a Kubernetes environment functions by listening for jobs from the GitLab instance and spawning temporary pods to execute those jobs. This architecture allows for high elasticity, as resources are only consumed during the actual execution of a CI/CD pipeline. To manage this deployment, the GitLab Runner Helm chart is utilized, offering configuration options across Free, Premium, and Ultimate tiers, and supporting GitLab.com, GitLab Self-Managed, and GitLab Dedicated offerings.

The core of the configuration resides in the values.yaml file, where the behavior of the runner and its associated build pods is defined. A critical component of this configuration is the use of configuration templates. These templates allow administrators to define the behavior of build pods without sharing specific runner configuration options directly with the Helm chart. This decoupling ensures that the environment remains flexible. Within the values.yaml file, the config: section must utilize TOML format (using the parameter = value syntax) because the config.toml file is embedded directly into the YAML structure.

Advanced Runner Configuration and Concurrency

One of the most vital settings for managing resource consumption and job throughput is the concurrency limit. The GitLab Runner is configured to handle a specific number of simultaneous jobs to prevent the underlying Kubernetes cluster from becoming overwhelmed.

The concurrent setting in the global section of the configuration determines how many jobs can run at once.

• Default Value: The system defaults to 10 concurrent jobs.
• Impact Layer: If the concurrency is set too low, pipelines will experience queuing delays, increasing the lead time for delivery. If set too high, the Kubernetes cluster may suffer from resource exhaustion, leading to pod eviction or node instability.
• Contextual Layer: This setting works in tandem with the Kubernetes cluster's autoscaling capabilities. As the runner initiates more concurrent jobs, the cluster must have sufficient CPU and memory overhead to satisfy the requests of the spawned build pods.

To configure this in the values.yaml file, the following syntax is used:

toml concurrent: 10

Privileged Containers and Docker-in-Docker (DinD)

Many CI/CD pipelines require the ability to build Docker images or manage containers within the job itself. This is typically achieved through Docker-in-Docker (DinD). However, running the Docker executable requires the container to have elevated privileges.

To enable this functionality, the runner must be configured to use privileged containers. This is achieved by modifying the [runners.kubernetes] section of the configuration.

• Configuration Step: In the values.yaml file, the privileged flag must be set to true.
• Impact Layer: Enabling privileged mode allows the container to access the host's kernel features and device drivers, which is necessary for the Docker daemon to function. However, this introduces a significant security risk, as it potentially allows a compromised job to escape the container and affect the host node.
• Contextual Layer: Because of the inherent risks associated with privileged mode, this configuration should only be used if the runner is registered against a specific project where the administrators trust the CI/CD jobs being executed.

The configuration implementation is as follows:

yaml runners: config: | [[runners]] [runners.kubernetes] privileged = true

Image Management and Private Registries

The ability to pull images from private registries is essential for enterprises that do not store their build environments in public repositories. This requires the configuration of imagePullSecrets.

To utilize an image from a private registry, the administrator must create one or more secrets within the specific Kubernetes namespace where the CI/CD jobs are executed. These secrets contain the credentials necessary for the Kubernetes kubelet to authenticate with the private registry. This ensures that proprietary build tools and secure base images can be deployed without exposing credentials in the pipeline scripts.

Security Contexts and Non-Root Execution

Security is a paramount concern in Kubernetes deployments. Running containers as the root user is generally discouraged. GitLab provides specific image variants and configurations to support non-root execution.

To implement a non-root user environment, the securityContext and the image selection must be aligned. This involves using specific UBI (Universal Base Image) variants designed for OpenShift and other hardened environments.

• Image Selection: The registry.gitlab.com/gitlab-org/ci-cd/gitlab-runner-ubi-images/gitlab-runner-ocp image is used.
• User Configuration: The runAsUser is set to 999.
• Helper Image Integration: The helper_image must also be specified to match the non-root requirements.

The detailed configuration for non-root execution is structured as follows:

yaml image: registry: registry.gitlab.com image: gitlab-org/ci-cd/gitlab-runner-ubi-images/gitlab-runner-ocp tag: v16.11.0 securityContext: runAsNonRoot: true runAsUser: 999 runners: config: | [[runners]] [runners.kubernetes] helper_image = "registry.gitlab.com/gitlab-org/ci-cd/gitlab-runner-ubi-images/gitlab-runner-helper-ocp:x86_64-v16.11.0" [runners.kubernetes.pod_security_context] run_as_non_root = true run_as_user = 59417

In this configuration, although run_as_user is pointed to the specific user ID 59417, the images are designed to work with any user ID. It is essential that this user ID is part of the root group, although being part of the root group does not grant additional specific privileges in this context.

FIPS Compliance and Specialized Images

For organizations operating in highly regulated environments, Federal Information Processing Standards (FIPS) compliance is required. This necessitates the use of FIPS-compliant images for both the runner and the helper.

To achieve FIPS compliance, the ubi-fips tag must be applied to the image and the helper_image_flavor must be explicitly set in the configuration.

• Implementation Step: Change the image tag to ubi-fips and update the helper_image_flavor in the [runners.kubernetes] section.
• Impact Layer: This ensures that the cryptographic modules used by the runner meet the strict standards required by government agencies.

The configuration for FIPS compliance is as follows:

yaml image: registry: docker.io image: gitlab/gitlab-runner tag: ubi-fips runners: config: | [[runners]] [runners.kubernetes] helper_image_flavor = "ubi-fips"

Ubuntu-Based Images and glibc Requirements

Certain build jobs require specific libraries, such as glibc, which are native to Ubuntu-based images. If the default images are insufficient, the runner can be switched to an Ubuntu-based image.

• Image Specification: Use gitlab/gitlab-runner:v17.3.0 or the latest tag.
• Security Context Adjustment: When switching to Ubuntu images, the securityContext must be updated to align with the user ID of the Ubuntu image, specifically setting fsGroup and runAsUser to 999.

The corresponding configuration is:

yaml image: gitlab/gitlab-runner:v17.3.0 securityContext: fsGroup: 999 runAsUser: 999

Caching and Object Storage Integration

To optimize pipeline performance, caching is utilized to store dependencies between jobs. When using the GitLab Runner Helm chart, this requires integration with an object storage provider.

• Configuration Variable: The runners.cache.secretName variable is used in the values.yaml file.
• Impact Layer: This secret points to the credentials of the object storage provider. Without this, every job must download all dependencies from the internet, significantly increasing build times and bandwidth consumption.

Deployment via AWS and Bitnami

For users seeking an alternative to the SaaS version of GitLab, deploying via Amazon Machine Images (AMI) or Bitnami-packaged applications on AWS provides a streamlined path. Bitnami provides pre-configured, secure, and up-to-date images for various applications, including GitLab, which can be deployed on Kubernetes.

The use of a Bitnami-provided AMI for GitLab can save significant amounts of installation and debugging time, particularly for those deploying on Debian or Amazon Linux. This is an efficient alternative to manual installation, as it provides a tested baseline. For additional support, AWS provides a 24x7x365 one-on-one support channel staffed by technical engineers to help customers utilize these features.

Integration with FOSSA for Compliance

A critical part of the CI/CD pipeline is the management of open-source license compliance and vulnerability management. This is achieved by integrating FOSSA into the GitLab CI/CD pipeline.

The integration involves the following components:
- .gitlab-ci.yml: The configuration file where the pipeline stages are defined.
- Pipeline Configuration: Defining when the FOSSA scan should occur (e.g., on every merge request).
- Vulnerability Management: Using FOSSA to automatically identify licenses and security vulnerabilities in dependencies.

Comparison of Configuration Parameters

The following table summarizes the key configuration parameters found within the GitLab Runner Helm chart and their primary functions.

Detailed Analysis of Deployment Strategies

The deployment of the GitLab Runner using the Bitnami and Helm approach represents a move toward "Infrastructure as Code" (IaC). By defining the entire environment in a values.yaml file, organizations can ensure that their build environments are reproducible and versioned.

The shift toward using non-root users and UBI images (specifically for OpenShift/OCP) is a response to the increasing security requirements of cloud-native environments. By restricting the runner's permissions and using a specific pod_security_context, administrators can mitigate the risk of container escape attacks. Furthermore, the ability to switch between glibc-based Ubuntu images and FIPS-compliant images allows the same Helm chart to be used across diverse regulatory environments.

The integration of the runner into the AWS ecosystem via Bitnami AMIs further reduces the "time to value." By removing the manual installation steps associated with Ruby on Rails and PostgreSQL (the backbone of GitLab), developers can focus on the .gitlab-ci.yml configuration rather than the underlying server maintenance.

The use of the config: block within the Helm chart to embed TOML is a specific technical requirement of the GitLab Runner. Because the runner's native configuration is written in TOML, the Helm chart acts as a wrapper, injecting this TOML block into the config.toml file on the pod's filesystem. This allows for a high degree of granularity in configuring the [runners.kubernetes] section, such as specifying the helper image or the pod security context.

Sources

1. GitLab Runner Kubernetes Helm Chart Configuration
2. Bitnami GitLab on AWS
3. AWS Marketplace - GitLab
4. FOSSA GitLab CI/CD Setup Guide