Optimizing GitHub Actions Workflows Through Granular File Change Detection

Published by TechMarmot Editorail Staff on | Category GitHub Actions

Introduction

In the modern software development lifecycle, Continuous Integration and Continuous Deployment (CI/CD) pipelines are the backbone of automated software delivery. However, as repositories grow in size and complexity, running every possible check, test, and build step for every minor commit becomes computationally expensive and time-consuming. A critical optimization technique for GitHub Actions is the implementation of conditional logic based on file changes. By detecting exactly which files have been modified—whether in a pull request, a push, or during the workflow execution itself—engineers can ensure that only relevant jobs are triggered. This approach not only conserves compute resources but also accelerates feedback loops for developers. The ecosystem provides several specialized actions to handle these scenarios, ranging from detecting changes relative to a target branch to verifying uncommitted changes generated during a workflow run. Understanding the nuances of tools like dorny/paths-filter, tj-actions/changed-files, tj-actions/check-for-changed-files, and tj-actions/verify-changed-files is essential for building robust, efficient, and intelligent CI/CD pipelines.

The Challenge of Conditional Workflow Execution

The foundational problem in CI/CD optimization is determining whether a specific set of files has changed before executing a job. A common pattern involves checking if source code files (such as Python scripts) have been modified before running the associated test suite. However, test suites are not solely dependent on source code; they are also affected by configuration files, dependency manifests, and the workflow definition itself.

Consider a scenario where a developer wants to run tests only if Python files or the test workflow configuration changes. An initial approach might involve checking for Python files and the workflow file separately. As the number of dependencies grows—such as tox.ini, requirements files, and test output fixtures—maintaining individual conditions becomes unwieldy. The logical solution is to group these file patterns into a single output variable that represents "files affecting tests."

The following configuration demonstrates this approach using dorny/paths-filter. It defines a job named changed that identifies modified files and outputs a boolean flag run_tests if any of the specified patterns match.

yaml jobs: changed: name: "Check what files changed" outputs: run_tests: ${{ steps.filter.outputs.run_tests }} steps: - name: "Check out the repo" uses: actions/checkout - name: "Examine changed files" uses: dorny/paths-filter id: filter with: filters: | run_tests: - "**.py" - ".github/workflows/testsuite.yml" - "tox.ini" - "requirements/*.pip" - "tests/gold/**"

Subsequent jobs can then depend on this changed job and use an if condition to execute only when necessary. Additional logic, such as skipping tests on branches named with -notests, can be integrated into the condition.

yaml tests: # Don't run tests if the branch name includes "-notests". # Only run tests if files that affect tests have changed. needs: changed if: | ${{ needs.changed.outputs.run_tests == 'true' && !contains(github.ref, '-notests') }}

This strategy centralizes file-change logic, making the workflow easier to maintain and update as the project evolves. It ensures that the test suite runs only when relevant files are modified, preventing unnecessary execution and saving CI/CD minutes.

Detecting Changes Relative to Branches and Commits

For more complex scenarios, such as tracking changes between tags, comparing a pull request to its base branch, or identifying changes in a specific subdirectory, the tj-actions/changed-files action provides a robust solution. This action is designed to track all changed files and directories relative to a target branch, the current branch (preceding commit or last remote commit), multiple branches, or custom commits. It returns relative paths from the project root, facilitating easy integration with subsequent steps.

Key features of tj-actions/changed-files include:

  • Fast execution, averaging 0-10 seconds.
  • Utilization of either GitHub's REST API or Git's native diff command.
  • Support for Git submodules and merge queues.
  • Generation of escaped JSON output for matrix jobs.
  • Capability to list changed directories with depth limitations.
  • Option to write outputs to .txt or .json files.

The action is particularly useful for release workflows, where one might need to identify changes between the previous tag and the current tag. The following example demonstrates how to retrieve all changed files between tags and process them.

```yaml
on:
push:
tags:
- 'v*'

jobs:
release:
name: Release
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0

  - name: Get changed files
    id: changed-files
    uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.5

  - name: List changed files
    env:
      ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
    run: |
      echo "List all the files that have changed: $ALL_CHANGED_FILES"

```

Furthermore, the action allows for targeted inspection of specific directories. For instance, to check if any file within the .github folder has changed, the files input can be specified.

```yaml
- name: Get changed files in the .github folder
id: changed-files-specific
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.5
with:
files: .github/**

  - name: Run step if any file(s) in the .github folder change
    if: steps.changed-files-specific.outputs.any_changed == 'true'
    env:
      ALL_CHANGED_FILES: ${{ steps.changed-files-specific.outputs.all_changed_files }}
    run: |
      echo "One or more files in the .github folder has changed."
      echo "List all the files that have changed: $ALL_CHANGED_FILES"

```

The action also supports repositories located in different paths within the runner, ensuring flexibility for mono-repos or projects with complex directory structures.

```yaml
- name: Checkout into dir1
uses: actions/checkout@v4
with:
fetch-depth: 0
path: dir1

  - name: Run changed-files with defaults in dir1
    id: changed-files-for-dir1
    uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.5
    with:
      path: dir1

```

It is important to note that tj-actions/changed-files solely identifies files that have changed for events such as pull_request, push, merge_group, and release. It does not detect pending uncommitted changes created during the workflow execution. For such scenarios, a different tool is required.

Enforcing File Change Requirements in Pull Requests

While detecting changes for conditional execution is common, another critical use case is enforcing that certain files must be changed in a pull request. This is often used to ensure that documentation, version numbers, or configuration files are updated alongside code changes. The tj-actions/check-for-changed-files action serves this purpose by verifying that specific files have been modified.

The action requires a glob pattern for the file(s) that must be changed. These patterns are matched using minimatch with the {dot: true} option, allowing for flexible matching. Multiple lines in the input are supported, with each line treated as its own glob pattern. For example:

yaml file-pattern: | package.json package-lock.json

This configuration acts as two separate patterns: package.json and package-lock.json. The action succeeds if any of the patterns match. Additionally, a pre-requisite glob pattern can be specified. If provided, the action proceeds only if this pattern matches; otherwise, it is considered successful. This allows for complex conditional logic where a check is only enforced if certain other files are also present or modified.

The action also supports forcing a skip of the check via a specific label on the pull request and allows for custom failure messages. These messages can utilize ${} syntax to insert input values, such as ${file-pattern}, ensuring clear feedback for developers. A GitHub auth token is optional but recommended for private repositories, typically using ${{ secrets.GITHUB_TOKEN }}.

Verifying Uncommitted Changes During Workflow Execution

In some cases, it is necessary to detect changes that occur during the workflow execution, such as files modified by a script or build step. The tj-actions/verify-changed-files action is specifically designed for this purpose. Unlike tj-actions/changed-files, which looks at historical commits, this action detects pending uncommitted changes generated in the current run.

Key capabilities include:

  • Fast execution, averaging 0-2 seconds.
  • Support for all major platforms (Linux, MacOS, Windows).
  • Compatibility with GitHub-hosted runners, GitHub Enterprise Server, and self-hosted runners.
  • Boolean output for detecting uncommitted changes.
  • Ability to list all files that changed during the workflow.
  • Detection of both tracked and untracked files.

The action supports advanced glob pattern matching, including globstar (**), brace expansion, and negation. This allows for precise control over which files are monitored. For example, one might want to check for changes in text files and a specific directory, while excluding SQL files.

yaml steps: - uses: actions/checkout@v4 - name: Change text file run: | echo "Modified" > new.txt - name: Change file in directory run: | echo "Changed" > test_directory/new.txt - name: Verify Changed files uses: tj-actions/verify-changed-files@a1c6acee9df209257a246f2cc6ae8cb6581c1edf # v20 id: verify-changed-files with: files: | *.txt test_directory action.yml **/*.{jpeg,py} !*.sql - name: Run step only when any of the above files change. if: steps.verify-changed-files.outputs.files_changed == 'true' env: CHANGED_FILES: ${{ steps.verify-changed-files.outputs.all_changed_files }}

This functionality is invaluable for ensuring that build processes do not inadvertently modify tracked files or for triggering additional steps based on the side effects of previous steps.

Handling Output Formatting and Security Considerations

When working with file lists in GitHub Actions, proper output formatting and security considerations are paramount. The tj-actions/changed-files action provides several options to manage how file lists are returned. By default, it escapes unsafe filename characters to prevent command injection vulnerabilities. However, if the output is stored in an environment variable and processed carefully, this escaping can be disabled.

```yaml
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.5
with:
safe_output: false # set to false because we are using an environment variable to store the output and avoid command injection.

  • name: List all added files
    env:
    ADDEDFILES: ${{ steps.changed-files.outputs.addedfiles }}
    run: |
    for file in ${ADDED_FILES}; do
    echo "$file was added"
    done
    ```

Additionally, the separator used between file names in the output can be customized. While a space is the default, a comma or other character can be specified to suit the needs of subsequent processing steps.

yaml - name: Get all changed files and use a comma separator in the output id: changed-files uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.5 with: separator: ","

Understanding these nuances allows developers to integrate file-change detection seamlessly into their workflows, ensuring both efficiency and security.

Conclusion

The ability to detect and respond to file changes is a cornerstone of efficient GitHub Actions workflows. By leveraging specialized actions like dorny/paths-filter, tj-actions/changed-files, tj-actions/check-for-changed-files, and tj-actions/verify-changed-files, developers can create sophisticated CI/CD pipelines that are both resource-efficient and highly responsive. Whether the goal is to skip tests on irrelevant commits, enforce documentation updates in pull requests, or verify uncommitted changes during a build, these tools provide the necessary building blocks. As repositories continue to grow in complexity, mastering these techniques will become increasingly important for maintaining fast, reliable, and scalable software delivery pipelines. The key lies in choosing the right tool for the specific context—whether looking back at historical commits or forward at runtime changes—and configuring it with precision to achieve the desired outcome.

Sources

  1. Ned Batchelder Blog
  2. tj-actions/changed-files
  3. tj-actions/check-for-changed-files
  4. tj-actions/verify-changed-files

Related Posts