Git Repository Structure for Microservices

Microservices architecture represents a paradigm shift in software engineering, transitioning from monolithic designs toward a collection of loosely coupled, independently deployable services. In this architectural model, each service is engineered to address a specific business capability and facilitates communication with other services via well-defined interfaces. While this modular approach grants immense flexibility, scalability, and resilience, it introduces significant complexities regarding source code management. The method chosen to structure repositories directly affects how teams collaborate, how deployment pipelines are executed, and how the system is maintained over its lifecycle. Because microservices are distributed by nature, the management of their source code requires a deliberate strategy to ensure that the independence of the services is not compromised by the structural overhead of the version control system.

Architectural Paradigms for Code Repositories

When establishing a codebase for a microservices environment, organizations generally gravitate toward two primary structural philosophies: the Monorepo approach and the Multirepo approach. The selection between these two is not merely a technical preference but a strategic decision that impacts the entire software development lifecycle.

The Monorepo Approach

A monorepo is a structural strategy where all microservices, shared libraries, and configuration files are stored within a single, unified Git repository. This centralized approach is designed to streamline several aspects of the development process.

• Cross-service changes: When a feature requires modifications across multiple services, a monorepo allows developers to commit these changes in a single atomic operation. This eliminates the need to coordinate multiple pull requests across different repositories.
• Dependency management: Managing dependencies is simplified because all services share a single version control history. This prevents the "version skew" that often occurs when different services rely on different versions of a shared internal library.
• CI Process Complexity: While simplified in some areas, monorepos can introduce significant complexity into continuous integration processes. As the repository grows, the CI system must be intelligent enough to trigger only the pipelines relevant to the changed code, rather than rebuilding the entire ecosystem for every commit.
• Version Control Overhead: Large monorepos can lead to slower Git operations (such as clones or fetches) and potentially more complex merge conflicts as the number of contributors increases.

The Multirepo Approach

In a multirepo architecture, every individual microservice is assigned its own dedicated Git repository. This is the antithesis of the monorepo and focuses heavily on autonomy and isolation.

• Independent Versioning: Each service can be versioned independently. This means a critical update to the "Payment Service" can be tagged and released without affecting the versioning state of the "User Service."
• Autonomous Team Workflows: Teams can manage their own repositories, choosing their own branching strategies and release cadences without being hindered by the constraints of other teams.
• Streamlined CI/CD Pipelines: Because each repository is small and focused, deployment pipelines are dedicated per service. This reduces the complexity of the pipeline logic and accelerates the deployment speed.
• Reduced Merge Conflicts: Smaller, focused repositories naturally result in fewer merge conflicts, as developers are only modifying code related to a specific business capability.
• Shared Dependency Challenges: The primary drawback of the multirepo approach is the difficulty in managing shared dependencies. Without a central repository, teams must be vigilant to avoid code duplication or inconsistent versions of shared logic across different services.

Git Submodules for .NET Microservices

A sophisticated alternative that bridges the gap between the monolithic and distributed approach is the use of Git Submodules. This is particularly effective in .NET environments, especially with the advancement of .NET Aspire. This strategy involves creating a "root" parent repository that acts as an orchestrator, pulling in various sub-projects as submodules.

The Mechanics of the Root Project

The root project serves as the entry point for the entire solution. Rather than containing the actual source code for every service, it contains references to other Git repositories. This structure allows a solution to encompass many sub-projects while maintaining the flexibility of individual repositories.

• Atomicity: Each project remains in its own individual Git repository, preserving the benefits of the multirepo approach.
• Solution Integration: The root project connects everything, allowing developers to see the overarching structure of the system.
• Visual Studio Support: This approach is highly supported within Visual Studio, enabling developers to step through code across different sub-modules efficiently.
• Troubleshooting Capability: It provides a powerful way to troubleshoot complex projects where code is shared across services.
• Branching Flexibility: Because each submodule is its own repository, teams have greater flexibility in their branching strategies for individual services.

Architectural Trade-offs of Submodules

Implementing a submodule-based structure involves specific trade-offs that must be weighed against the team's capabilities and tools.

• Complexity Level: This approach is more complex and is not recommended for beginners. It requires a higher level of Git command line knowledge to manage effectively.
• Tooling Support: While Visual Studio offers strong support, other integrated development environments (IDEs) may have less robust support for Git Submodules, potentially hindering productivity.
• Docker Build Contexts: A significant technical challenge is that Docker build contexts are relative. When using multiple Dockerfiles across submodules, managing the context for builds becomes more difficult.
• Git Proficiency: This structure can be confusing for developers who do not possess a deep understanding of Git, potentially leading to errors in how submodules are updated or committed.
• DLL and Library Risks: Using reusable libraries via NuGet packages in this context can be risky. If not managed correctly, it can lead to a "DLL nightmare" where unnecessary dependencies create a complex web of versioning conflicts.

Implementation Guide for Git Submodules

Integrating submodules requires a sequence of specific Git commands to ensure the parent repository and the sub-projects are correctly linked.

Adding Submodules to a Project

When adding a submodule, it is critical to specify the branch and the target directory. This ensures the project remains organized and easy to navigate.

• The -b flag: This specifies the branch being added to the repository (e.g., main).
• Trailing path: The path specified at the end of the command (e.g., code/Common) determines the folder where the submodule will be placed.

To add a common shared project (DLL), the following commands are used:

git submodule add -b main [email protected]:mrjamiebowman-blog/microservices-projectstructure-common.git code/Common

Following the addition, the submodules must be updated:

git submodule update --remote

To add an API project:

git submodule add -b main [email protected]:mrjamiebowman-blog/microservices-projectstructure-api.git code/Api

Then update:

git submodule update --remote

To add a Web project:

git submodule add -b main [email protected]:mrjamiebowman-blog/microservices-projectstructure-web.git code/Web

Then update:

git submodule update --remote

Once these operations are complete, the root repository will contain references to the submodules. These references are stored as shortened SHA-1 hash IDs, which point to the exact state of the sub-project in its respective repository.

Cloning a Root Project

Developers joining a project that utilizes submodules cannot use a simple git clone if they want the sub-projects to be populated. There are two primary methods to handle this.

Method 1: Recursive Clone

This is the most efficient method as it clones the root repository and all its submodules in a single operation.

git clone --recurse-submodules [email protected]:mrjamiebowman-blog/microservices-projectstructure-root.git cd microservices-projectstructure-root

Method 2: Clone and Update

If the repository has already been cloned without the recursive flag, the submodules must be initialized and updated manually.

git clone [email protected]:mrjamiebowman-blog/microservices-projectstructure-root.git cd microservices-projectstructure-root

To initialize the submodules:

git submodule update --init --recursive

To update all submodules to their latest remote state:

git submodule update --recursive --remote

Best Practices for Microservice Repository Structuring

Regardless of whether a team employs a monorepo, a multirepo, or a submodule-based approach, certain architectural standards must be applied to ensure long-term maintainability.

Domain-Driven Design (DDD) Integration

Repositories should be structured around clearly defined business domains. By aligning the code structure with the business domain, teams can maintain a logical separation of concerns and simplify the boundaries between services. This prevents the leaking of logic from one domain into another, which would otherwise lead to a "distributed monolith."

Standardization and Navigation

To facilitate rapid onboarding and efficient code navigation, a consistent repository structure is mandatory.

• Folder Standardization: Every repository should follow the same internal folder hierarchy. If one service uses src/ for source code and tests/ for testing, every other service must do the same.
• Naming Conventions: Repositories should use descriptive and standardized naming conventions. Names should clearly reflect the purpose of the service (e.g., ordering-service instead of project-1).
• Comprehensive Documentation: Each repository must contain detailed documentation. This should include:
  • Setup instructions for local development.
  • List of dependencies.
  • API specifications.
  • Deployment guidelines.

Advanced Dependency Management

Managing how microservices depend on one another and on shared code is one of the most critical aspects of the architecture.

• Package Managers and Registries: To manage shared libraries without duplicating code, teams should utilize package managers such as npm or Maven. These should be paired with private artifact registries to ensure that shared internal libraries are versioned and distributed securely.
• Semantic Versioning: A strict semantic versioning policy must be implemented. This ensures that breaking changes are communicated through version numbers, allowing dependent services to upgrade at their own pace without causing system-wide failures.
• Automated Scanning: Regularly scanning dependencies for vulnerabilities and outdated versions is essential to maintain security and stability.

Comparison of Repository Strategies

The following table provides a detailed comparison of the three primary strategies discussed: Monorepo, Multirepo, and Git Submodules.

Analysis of Structural Selection

The decision regarding repository structure is not a one-size-fits-all solution but a calculation based on team maturity, the complexity of the domain, and the tools available. The Monorepo approach is often superior for smaller teams or projects in the early stages of development where cross-service changes are frequent and the overhead of managing multiple repositories would outweigh the benefits of isolation. However, as an organization scales, the Monorepo can become a bottleneck, leading to slow build times and "merge hell."

The Multirepo approach is the ideal choice for large, distributed organizations where teams are fully autonomous. It minimizes the blast radius of changes and allows for a highly optimized CI/CD pipeline. Yet, it places a heavy burden on the organization to manage shared libraries through artifact registries to avoid the "DLL nightmare" described in the .NET context.

The Git Submodule approach, particularly within the .NET ecosystem, offers a hybrid solution. It provides the atomicity of the multirepo—where each project is its own entity—while offering the organizational benefits of a monorepo through the root project. This allows a developer to treat the entire microservices ecosystem as a single unit for the purpose of troubleshooting and local development, while still maintaining the ability to version and deploy services independently. However, the cost of this flexibility is a steep learning curve and a requirement for high Git proficiency.

Ultimately, the choice must reflect the current state of the team. One must consider the editors being used, the skill level of the developers, and what provides the most productivity at the current moment. The transition from a simple structure to a more advanced one, such as submodules, should be driven by the need for greater flexibility in branching strategies and the necessity of troubleshooting complex, shared-code projects.

Sources

1. Mr. Jamie Bowman
2. Harness DevOps Academy