Understanding a CI/CD pipeline

The development life cycle of software systems goes from code, through testing, generating artifacts, even more testing, and eventually, deployment to production. The basic idea is that whenever a developer commits changes to their source control system (for example, GitHub), these changes are detected by the continuous integration (CI) system, which immediately runs the tests.

This is often followed by a review by peers and merging the code changes (or a pull request) from a feature branch or development branch into the master. In the context of Kubernetes, the CI system is also responsible for building the Docker images for the services and pushing them to the image registry. At this point, we have Docker images that contain new code. This is where the CD system comes in.

When a new image becomes available, the continuous delivery (CD) system will deploy it to the target environment(s). CD is the process of ensuring that the overall system is in a desired state, which is done though provisioning and deployments. Sometimes, deployment can occur as a result of configuration change if the system doesn't support dynamic configuration. We will discuss configuration in great detail in Chapter 5, Configuring Microservices with Kubernetes.

So, a CI/CD pipeline is a set of tools that detect code changes and can take them all the way to production according to the processes and policies of the organization. It is typically the responsibility of DevOps engineers to build and maintain this pipeline, and it is used heavily by developers.

Every organization and company (or even different groups within the same company) will have a specific process. In one of my first jobs, my first task was to replace a Perl-based build system (that's what CI/CD pipelines were called back then) with lots of recursive makefiles that nobody understood any more. That build system had to run code generation steps on Windows using some modeling software, compile and run C++ unit tests on two flavors of Unix (including an embedded flavor) using two different toolchains, and trigger open CVS. I chose Python and had to create everything from scratch.

It was fun, but very specific to this company. It's common to think of CI/CD pipelines as a workflow of steps driven by events.

The following diagram demonstrates a simple CI/CD pipeline:

The stages in this pipeline function as follows:

1. The developer commits their changes to GitHub (source control)
2. The CI server runs the tests, builds a Docker image, and pushes the image to DockerHub (image registry)
3. The Argo CD server detects that there is a new image available and deploys to the Kubernetes cluster

Now that we have understood the CI/CD pipeline, let's examine a specific CI/CD pipeline choice.