Skip to content

Commit

Permalink
ci: avoid running the test suite _twice_
Browse files Browse the repository at this point in the history
This is a late amendment of 19ec39a (ci: stop linking the `prove`
cache, 2022-07-10), fixing a bug that had been hidden so far.

The bug is that the `.prove` cache stores information about previous
`prove` runs (`save`) and uses them (`slow`, to run the tests in the
order from longer-running to shorter ones).

This bug can cause some surprising behavior: when the Prove cache
contains a reference to a test script, subsequent `prove` runs (with
`--state=slow`) will run the same test script again even if said script
is not specified on the `prove` command-line!

So far, this bug did not matter. Right until d8f416b (ci: run unit
tests in CI, 2023-11-09) it did not matter.

But starting with that commit, we run `prove` _twice_ in CI, and with
completely different sets of tests to run. Due to the bug, the second
invocation re-runs all of the tests that were already run as part of the
first invocation. This not only wastes build minutes, it also frequently
causes the `osx-*` jobs to fail because they already take a long time
and now are likely to run into a timeout.

The worst part about it is that there is actually no benefit to keep
running with `--state=slow,save`, ever since we decided no longer to
try to reuse the Prove cache between CI runs.

So let's just drop that Prove option and live happily ever after.

Signed-off-by: Johannes Schindelin <[email protected]>
  • Loading branch information
dscho committed Nov 13, 2023
1 parent 309e3ec commit d5b753d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion ci/lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ else
fi

MAKEFLAGS="$MAKEFLAGS --jobs=$JOBS"
GIT_PROVE_OPTS="--timer --jobs $JOBS --state=failed,slow,save"
GIT_PROVE_OPTS="--timer --jobs $JOBS"

GIT_TEST_OPTS="$GIT_TEST_OPTS --verbose-log -x"
case "$CI_OS_NAME" in
Expand Down

0 comments on commit d5b753d

Please sign in to comment.