Explore npm ci for CI/CD

[bdemann]

We should evaluate the benefits of using npm ci instead of npm install for our automated tests. While we’ve currently opted to stick with npm install to simulate how users are likely to install dependencies, we could achieve this simulation specifically during the npm tests. For the repo tests, npm ci might provide significant advantages.

However, adopting npm ci comes with some considerations:

1. Package sync errors: It fails if package.json and package-lock.json are out of sync. While this behavior could be desirable in the future, we would need a script or workflow to synchronize these files before enforcing it.

Benefits of using npm ci:

1. Immutable package files during tests: Since we don’t commit any artifacts during tests, npm ci guarantees that package.json and package-lock.json remain unchanged.
2. Clean installation: By removing the node_modules folder before installation, it ensures a clean, reproducible state. While our CI processes typically run on new instances (avoiding pre-existing node_modules folders), this reinforces best practices.
3. Up-to-date package-lock.json: Encourages keeping package-lock.json in sync with package.json.
4. Deterministic builds: It strictly adheres to the exact versions in package-lock.json, reducing variability in dependency resolution.
5. Performance improvements: npm ci is faster since it skips certain features intended for interactive environments. Although installation speed isn’t a bottleneck for us, even small optimizations can add up.

Next Steps

Let’s explore the feasibility and impact of using npm ci, including any adjustments we might need to enforce proper synchronization of package.json and package-lock.json.

[bdemann]

If we decide to adopt npm ci, here’s how I’d propose updating the workflows:

1. benchmark.yml and benchmark_parallel.yml: Replace the npm install commands with npm ci.
2. run_test.yml: Update to use npm ci by default. However, if link azle is false, fallback to npm install instead.
3. release.yml: Leave this workflow unchanged. Since we update package.json during the release process, we’ll also want to update package-lock.json and commit those artifacts as part of the release.