Each run of test.sh is very slow

[carlosgmartin]

On my machine, each run of sh test.sh takes about 4.5 minutes, which is inconvenient for development.

One potential way to make it faster could to keep a gitignored test_venv directory at the root of the repo that persists from run to run (that is, is not cleaned up by test.sh), so that all the pip install packages used by test.sh don't need to be reinstalled on each run.

Doing this brings the runtime down to 1 minute on my machine (after the initial run that installs all the packages). I can submit a PR demonstrating the changes, if desired. I'm also happy to hear any other suggestions.

[vroulet]

I am surprised that the issue could be the installation of packages.
To me what slows down the tests a lot is (i) pytype, (ii) doc building and doctests.
However, by keeping the test_venv, some of theses processes (like ptype) may keep a cache that accelerates subsequent tests.

In any case, if you have a solution that brings the tests down to 1 minute locally, we may at least add an option like a --local flag to prevent the cleanup and add the test_venv to the gitignored files.

@fabianp may also know better on such issues.

[carlosgmartin]

@vroulet Yes, that kind of caching might be contributing to the speedup as well.

[fabianp]

we could also have different scripts for {pytype, tests, building and testing documentation}, this way they could be run in parallel in github actions

[carlosgmartin]

@fabianp @vroulet I've suggested some related changes at #1110.