This article will summarize the different steps we have taken to enhance our CI when working with Rust.
- Parallelize Run
- Simplify the CI jobs
- Caching build artifacts
- Optimize CI build
- Verify code format and lint
- Run Test
With our previous CI on Azure Pipelines, we ran the same job on different platforms (Windows/Linux/macOS). The job involved running tests and validating code style for both the Python and Rust codebases.
With the migration, we split the monolithic job into multiple smaller jobs:
- One job for testing the Python code on all platforms.
- Another job for testing the Rust code.
- A job to verify the quality and style of anything other than Python and Rust.
Having multiple jobs allows for parallel execution.
Instead of testing the Python code and then the Rust code sequentially, we can now test both of them simultaneously, saving time on the feedback loop for developers.
Simplify the CI jobs even further
With the migration and the split of our monolithic job into multiple smaller jobs, we also took advantage of the myriad of GitHub Actions available to factorize common steps.
Common steps could include:
- Installing a specific Python version.
- Downloading cache artifacts and then uploading them once the job is finished.
- Running pre-commit.
- Uploading artifacts.
Caching build artifacts
On the Default GitHub Runner, compiling the Rust artifacts takes some time (~5 min/40 sec without/with caching). Since not all artifacts are modified with each pull request, it makes sense to cache them to reduce the amount of code that needs to be compiled.
We use the ready-made action Swatinem/rust-cache to cache the Rust artifacts. This action smartly caches only the third-party dependencies that don't change outside of updates.
Since cache entries on GitHub are shared from the main branch to the child branches (i.e., a branch originating from the main branch can use the cache entries of the main branch), we choose to save the entries only on the main branch.
All of this is done to save the maximum amount of data on the limited GitHub cache, allowing for an increased number of branches that can use the cache without GitHub invalidating it.
When you use more cache than available, GitHub starts removing old
entries until it returns within the 10 GB limit.
See: 10 GB of cache
Optimize CI build
In addition to caching the Rust artifacts, we also configured the cargo compilation profile. We created two new profiles used in our CI and
modified the default
dev profile, we modified the level of optimization for our third-party dependencies to:
opt-level = 1
This allows us to run the tests faster at the cost of increased compilation time, but it's a good trade-off since the Rust artifacts are cached.
In addition to modifying the
dev profile, we have created two new profiles that inherit from
inherits = "dev"
debug = false
incremental = false
inherits = "ci-rust"
opt-level = 1
The first profile
ci-rust is used by the CI part that runs/builds/tests the Rust code. It disables the
debugdisabled, no debug info is generated (like on a
releasebuild). We chose to disable them since it reduces the size of the generated artifacts (and the amount of generated data per se), resulting in reduced cache usage. In case of a test failure on the CI, we don't have a precise traceback, but we consider it is always simpler to debug it on the development machine.
- We toggle off the
incrementalsettings since we don't cache the artifacts for our internal crates1. This simplifies compilation and reduces the artifact size.
The second profile
ci-python is almost identical to
ci-rustexcept that it has a slightly optimized build for our Rust artifacts (
opt-level = 1) as they have an impact
on the Python tests' speed.
Verify code format and lint
cargo-clippy requires compilation artifacts, we have decided to execute these steps inside the CI part dedicated to Rust. This allows us to cache those artifacts and also prevents running unnecessary steps (if we don't modify Rust code, we don't need to run those steps).
We have chosen to use
cargo-clippy through the
pre-commit because currently we cannot easily configure
clippy for the entire workspace. This limitation will be addressed when cargo#12115 is implemented, possibly in the
cargo-nextest instead of
cargo-test to run our tests because:
- It provides a nicer output than the default
- It can be configured to abort tests that take too long to finish.
- It allows us to retry flaky tests2.
However, it's important to note that
nextest has a specific execution model where each test runs in a dedicated process (https://nexte.st/book/how-it-works.html). This means that shared memory between these tests is not possible.