You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, all the project's tests are done sequentially by pytest, and take about ~1.5 hours to run. Running tests in parallel would improve this, but since some tests run on hardware it's not as simple as installing pytest-xdist and calling it a day.
Ideally, there would be parallel execution in three groups:
Xilinx-based HW tests, which would all run sequentially in a single thread.
ice40-based HW tests, which (as above) would all run sequentially in a single thread.
Simulation-based tests, which would dispatch as many threads as desired. My best guess is that this would be equal to the number of cores remaining on the host machine after the Xilinx and ice40 tests claim their threads, but if any of the tools try to do anything multi-threaded this might cause some unnecessary CPU oversubscription.
It would be really cool if the build/test stages could be decoupled, so that bitstreams could be built while the hardware is occupied. This might be difficult for the Amaranth-based HWITL tests, since the RTL and the test are specified in the same class.
It looks like coverage.py easily handles merging coverage reports, so If this requires multiple pytest invocations then preserving coverage collection shouldn't be an issue.
The text was updated successfully, but these errors were encountered:
Currently, all the project's tests are done sequentially by
pytest
, and take about ~1.5 hours to run. Running tests in parallel would improve this, but since some tests run on hardware it's not as simple as installingpytest-xdist
and calling it a day.Ideally, there would be parallel execution in three groups:
It would be really cool if the build/test stages could be decoupled, so that bitstreams could be built while the hardware is occupied. This might be difficult for the Amaranth-based HWITL tests, since the RTL and the test are specified in the same class.
It looks like
coverage.py
easily handles merging coverage reports, so If this requires multiplepytest
invocations then preserving coverage collection shouldn't be an issue.The text was updated successfully, but these errors were encountered: