Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize/Optimize Tests #30

Open
fischermoseley opened this issue Dec 5, 2024 · 0 comments
Open

Parallelize/Optimize Tests #30

fischermoseley opened this issue Dec 5, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@fischermoseley
Copy link
Owner

Currently, all the project's tests are done sequentially by pytest, and take about ~1.5 hours to run. Running tests in parallel would improve this, but since some tests run on hardware it's not as simple as installing pytest-xdist and calling it a day.

Ideally, there would be parallel execution in three groups:

  • Xilinx-based HW tests, which would all run sequentially in a single thread.
  • ice40-based HW tests, which (as above) would all run sequentially in a single thread.
  • Simulation-based tests, which would dispatch as many threads as desired. My best guess is that this would be equal to the number of cores remaining on the host machine after the Xilinx and ice40 tests claim their threads, but if any of the tools try to do anything multi-threaded this might cause some unnecessary CPU oversubscription.

It would be really cool if the build/test stages could be decoupled, so that bitstreams could be built while the hardware is occupied. This might be difficult for the Amaranth-based HWITL tests, since the RTL and the test are specified in the same class.

It looks like coverage.py easily handles merging coverage reports, so If this requires multiple pytest invocations then preserving coverage collection shouldn't be an issue.

@fischermoseley fischermoseley changed the title Parallelize Tests Parallelize/Optimize Tests Dec 5, 2024
@fischermoseley fischermoseley added the enhancement New feature or request label Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant