Automating Continuous Deployment via GitLab CI/CD to Remote and Local Infrastructure

The transition from manual file transfers to automated deployment pipelines represents a fundamental shift in the software development lifecycle. In a modern DevOps ecosystem, the objective is to eliminate human error and latency by ensuring that code changes move from a developer's workstation to a production or staging server with minimal intervention. GitLab, an open-source collaboration platform, provides a robust framework for this through its CI/CD (Continuous Integration/Continuous Deployment) capabilities. Unlike simple version control, GitLab CI/CD introduces automation into the various stages of app development, allowing for frequent and reliable delivery of applications to customers.

At the heart of this automation is the GitLab Runner, an application written in the Go language designed to execute the specific jobs defined in a project's pipeline. The Runner acts as the execution engine, picking up instructions from the GitLab server and running them on a host machine—which could be a local server, a virtual private server (VPS), or a containerized environment. By configuring these Runners and defining the logic within a .gitlab-ci.yml file, engineers can transform a simple code push into a complex, multi-stage deployment process involving building, testing, and finally, deploying code to specific directory paths on a target server.

The Architecture of GitLab CI/CD and the Role of the Runner

To understand how to deploy code, one must first master the relationship between the GitLab instance and the Runner. While the GitLab server manages the orchestration, the Runner performs the actual heavy lifting.

The Runner is an open-source component that can be installed on a wide variety of supported operating systems. This versatility is critical because it allows organizations to run deployment jobs on the same hardware where the application resides or on entirely separate dedicated deployment machines.

Runner Execution Models

The method by which a Runner executes a job significantly impacts the deployment strategy and the complexity of the environment configuration.

• Shell Runner
  The Shell Runner is the most direct method for deploying to a local server. It executes commands directly on the host machine's shell. This is particularly useful for "provisioning" or "deploying" where the goal is to run commands like git pull or database migrations directly on the target machine. However, using a Shell Runner requires careful management of permissions and environment dependencies, as the commands run with the privileges of the user assigned to the Runner.

• Docker Runner
  As opposed to the Shell Runner, the Docker Runner executes jobs within isolated containers. This provides a clean, consistent environment for every job, preventing "dependency hell" where one job's requirements conflict with another. Using containers is highly recommended because it avoids the need to manually check if specific software is installed or to manually clean up databases and schemas after a job completes.

• Docker-in-Docker (DinD)
  In sophisticated on-premise infrastructures, specifically those operating offline, engineers often utilize a Docker-in-Docker image as the GitLab Runner image. This allows for the use of docker-compose or standard docker commands within the pipeline to connect various services together, facilitating complex microservices architectures.

Comparison of Runner Deployment Strategies

Infrastructure Setup and Runner Installation on Ubuntu

Before a pipeline can execute, the infrastructure must be prepared. For an Ubuntu-based server intended to act as a deployment target, the GitLab Runner must be installed and registered.

Step-by-Step Installation Procedure

The following sequence outlines the technical requirements for deploying a Runner on an Ubuntu system.

1. Add the official GitLab repository to the server to ensure access to the correct binaries and updates.
2. Download the specific binary for the system architecture using the following command:
   sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64" |
3. Grant execution permissions to the downloaded binary:
   sudo chmod +x /usr/local/bin/gitlab-runner
4. Create a dedicated system user to run the service. This user, typically named gitlab-runner, should have its own home directory and a bash shell:
   sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
5. Install the Runner as a system service, specifying the user and the working directory:
   sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
6. Configure the sudoers file to allow the gitlab-runner user to execute commands without a password prompt. This is a critical step because the automated pipeline cannot respond to an interactive password request. Use vi /etc/sudoers to add the necessary NOPASSWD permissions.
7. Start the service:
   sudo gitlab-runner start
8. Verify the service status:
   service gitlab-runner status

Security and Permissions Management

A major hurdle in automated deployment is the permission layer. If the Runner user (the user executing the shell commands) does not have the appropriate permissions to write to the webserver's directory or execute sudo commands, the deployment will fail.

• Sudo Access: As noted in the installation process, the gitlab-runner user must be granted NOPASSWD permissions in the /etc/sudoers file. Without this, any command requiring elevated privileges (like sudo rm or sudo cp) will hang indefinitely while waiting for a password that will never be entered.
• SSH Keys and Deploy Keys: For remote deployments where the Runner is not on the same machine as the webserver, the Runner must have a way to authenticate. This is typically achieved through pre-shared SSH keys. GitLab provides specific documentation for managing SSH keys and deploy keys to ensure the Runner can securely log into the target server and execute deployment scripts.

Orchestrating the Deployment via .gitlab-ci.yml

The .gitlab-ci.yml file is the configuration blueprint that defines the entire pipeline. This file resides in the root of the repository and tells GitLab exactly what to do when code is pushed or merged.

Pipeline Configuration Components

Within the .gitlab-ci.yml file, several key elements can be defined to control the flow of the application:

• Stages: These define the order of execution (e.g., build, test, deploy).
• Scripts: The actual shell commands to be executed.
• Dependencies and Caches: Used to speed up jobs by reusing files from previous stages.
• Deployment Location: Specifying where the application files should end up.
• Execution Triggers: Defining whether a job runs automatically (e.g., on a merge to main) or requires a manual trigger.

Implementing an Automated Deployment Workflow

A common workflow involves having a staging branch and a main branch. Developers push code to the staging branch. Once the code is verified, it is merged into the main branch. The act of merging into main triggers the deployment job.

The following example demonstrates a configuration where files are deployed to a specific directory on the server:

```yaml
stages:
- build
- deploy

build-job:
stage: build
script:
- echo "Compiling the code..."
- echo "Compile complete."

deploy-job:
stage: deploy
tags:
- server1-tag
only:
- main
script:
- echo "Starting deployment..."
- sudo rm -rf /home/ubuntu/*
- sudo cp -r * /home/ubuntu/
```

In this configuration:
- The stages block defines the sequence of the pipeline.
- The tags attribute ensures the job only runs on a Runner that has the server1-tag assigned to it.
- The only: - main directive ensures that the deployment only occurs when changes are made to the main branch, preventing accidental deployments from feature or staging branches.
- The script block contains the destructive and constructive commands: sudo rm -rf /home/ubuntu/* clears the existing directory, and sudo cp -r * /home/ubuntu/ copies the new files into the target location.

Optimization with GIT_STRATEGY

In deployment-only jobs, the Runner often does not need to clone the entire repository if the files are already present or if the deployment is handled via different methods (like pulling from a remote repo). In such cases, setting GIT_STRATEGY: none can optimize the job by skipping the fetch/clone phase, saving time and bandwidth.

Advanced Deployment Methodologies

While the simple "copy-paste" method works for basic web pages, professional environments require more sophisticated approaches.

The Pull-Based Deployment

For local servers, a Shell Runner can be used to execute a git pull command. This is often cleaner than copying files manually:

1. Navigate to the application directory:
   cd /path/to/application
2. Pull the latest code from the repository:
   git pull && git checkout <tagname>
3. Execute necessary updates:
   run some kind of update and/or database migrations script.

Provisioning vs. Deploying

It is vital to distinguish between these two concepts in a CI/CD context:
- Provisioning: The act of preparing the server or VPS, such as installing MySQL, Apache, or PHP. This is often handled via tools like Terraform or Ansible, but can be part of a CI/CD pipeline.
- Deploying: The act of installing or updating the specific application code onto the already provisioned server.

Technical Summary of Deployment Variables

The following table summarizes the critical configuration points for a GitLab CI/CD deployment pipeline.

Analysis of Deployment Security and Reliability

The move toward automated deployment via GitLab CI/CD introduces a shift in the security perimeter. When a Runner is granted sudo access or SSH access to a production server, the GitLab Runner becomes a high-value target. If the GitLab instance or the specific repository is compromised, the attacker gains a direct path to the server through the automated pipeline.

To mitigate these risks, engineers must implement strict "least privilege" policies. Instead of granting broad sudo access, it is better to grant specific permissions for only the necessary directories. Furthermore, the use of SSH keys should be restricted to "Deploy Keys," which are read-only or limited to specific tasks, rather than using full user credentials.

From a reliability standpoint, the distinction between Shell Runners and Docker Runners is the most significant technical decision a DevOps engineer will make. While Shell Runners are easier to set up for local deployments, they lack the "idempotency" and isolation provided by Docker. In a containerized world, the ability to ensure that every deployment starts from a known, clean state is paramount to avoiding the "it works on my machine" phenomenon. The complexity of managing dependencies manually on a host machine (the Shell Runner approach) often outweighs the initial setup time required to implement a containerized (Docker) workflow.

Ultimately, successful GitLab CI/CD deployment is not merely about writing a script to move files; it is about constructing a repeatable, secure, and isolated pipeline that bridges the gap between code development and live application availability.

Sources

1. GitLab Forum: How to use GitLab to deploy files directly on the webserver
2. GitLab Forum: GitLab CI deployment help
3. Cloudkul: Deploy code from GitLab to server using GitLab CI/CD