A common CI pipeline looks like this:
These are common stages.
Build the code -> Pre-check the code (quality checks like linting etc.) -> Test the code -> Deploy the code
Let’s look deeper into the testing stage:
-> Unit tests -> Integration tests -> Acceptance tests (system tests) -> Performance tests
These tests can be even more sophisticated and have more jobs that run in parallel.
The way a CI pipeline works is that once a stage fails, the next stage will not be executed—the build has failed and the team will be notified of this failure. They fix the problem, commit the changes and a new build, a new pipeline starts. Then they can see whether or not they fixed it.
Depending on your build, pre-check and test stages this whole process can take many minutes. I have seen pipelines that run for 30-50 minutes. I have heard of pipelines that run for hours.
Imagine that you have one acceptance test that fails, that is run rather late in the testing stage. You will spend many minutes (or hours) to learn that this test has failed. It is an important test, for a part of the application that is frequently touched by many developers. If one or more of your tests have this much importance, you should handle them with a higher priority! After all, if these tests fail you have to fix them before you can proceed to later stages in the pipeline.
That’s why I advocate introducing another stage into your pipeline:
Build the code -> Run high-prio tests -> # This is the new stage Pre-check the code (quality checks like linting etc.) -> Test the code -> Deploy the code
This makes your pipeline fail earlier and give back your precious time to fix things and get them back into a working state.
The idea of having a high-priority test stage ties neatly into the risk management of your project. If you identified risks in your project that are related to code, they are a good candidate to include in these earlier tests.