Should CI/CD scripts be in the app repo?

When you are developing an in-house application such as an ecommerce web app or a data pipeline, and adopting DevOps practices, the question can arise as to where CI/CD scripts should live: in the repo with the application source code, or somewhere else?

Broadly, you need to separate the “software” product from the “service” product, at least conceptually.

Even if your software product only has customer (you), failing to draw a proper boundary around it will bite you later.

The “software” product, which clearly does live in the app repo, should contain:

• build scripts, whether they be Dockerfiles, Makefiles, Azure Pipelines YAMLs or whatever build system you require or support for putting the software in a distributable form and placing it in a repository;
• sample deployment scripts for how a hypothetical third-party customer of the software product (a customer of the software, not a customer of the service) might deploy that software in their own environment.

The “service” product is different; this is your actual production instance of the software, running to serve your users. The “CD” part of CI/CD is how you take the built, distributable form of the software and get it in front of production traffic. (Non-production environments are, of course, simply duplicates of the production environment, perhaps with slightly different configuration settings.)

These production deployment scripts should be decoupled from any particular build of the software. In contrast, build scripts absolutely need to be coupled to the version of the software they are building.

This issue is particularly important when using build systems like Azure Pipelines (part of Azure DevOps) which allow you to define build pipelines outside of any particular repo. Doing so creates a problem when two versions of the software need to be built concurrently, perhaps one on the master branch and another on a feature branch that requires a different build process. Friction in this scenario means a brake on the evolution of your CI/CD system, and you adopted CI/CD because you want to go fast not slow, because you want things to be smooth not frustrating.

Photo by Josue Isai Ramos Figueroa on Unsplash

The deployment scripts, instead, are ideal candidates for living somewhere in your environment that isn’t necessarily a source code repo. Generally speaking you will not fork and merge deployment scripts or their associated configuration. Instead, the significance of “version control” over deployment scripts is related to audit and logging: at a particular point in time in the past, which version of our config was running? To the extent that a deployment script is under active development, this should occur under the auspices of developing the sample deployment scripts that exist within the app repo.

If the real deployment scripts are in a git repo, they should be separated from the software product. The deployment scripts can be stored in the app repo only on the understanding that you now have two distinct products living in a single repo and need to apply monorepo management techniques. Each product has its own purpose and its own lifecycle. Conflating the two seems OK at first, but it will lead to friction during times of architectural change (even minor changes), when errors arise in the CI/CD scripts themlselves, and when trying to roll back failed deployments. These are the very times that you can least afford friction, the times when you most need to lean on your build and deploy scripts.

A simple approach for getting started quickly is to use a single “CI/CD” script which now only builds the source code but also takes it all the way through testing, staging and production deployments. However, it should be a priority to logically divide the process into “build” scripts (versioned alongside the code it builds) and “deployment” scripts (updated according to the business and IT needs for deployed environments).