Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: parallel pytest execution #2131

Closed
wants to merge 12 commits into from
Closed

ci: parallel pytest execution #2131

wants to merge 12 commits into from

Conversation

sylvlecl
Copy link
Member

@sylvlecl sylvlecl commented Sep 2, 2024

Experiment the use of pytest-xdist to speed up test execution in CI and dev environment:
see for example this job where tests are executed in under 10min on ubuntu.

Following issues with parallel execution of tests are solved :

  • some checks depended on the consistency at 2 moments in time of the result of disk_usage, which was anyway not a good idea (dependent on what happens elsewhere on the machine). It's now replaced with mocking
  • some parameterized tests had random sort order, which is not allowed by pytest-xdist (all workers must have exactly the same tests in the same order)
  • the command_factory test checked something in its teardown, assuming all tests were executed in the same process, which is not the case anymore. The same check is done as a separate unit tests.

Signed-off-by: Sylvain Leclerc <[email protected]>
Signed-off-by: Sylvain Leclerc <[email protected]>
Signed-off-by: Sylvain Leclerc <[email protected]>
@pull-request-size pull-request-size bot added size/S and removed size/XS labels Sep 2, 2024
@pull-request-size pull-request-size bot added size/M and removed size/S labels Sep 2, 2024
@pull-request-size pull-request-size bot added size/XL and removed size/M labels Sep 3, 2024
@sylvlecl sylvlecl requested a review from mabw-rte September 3, 2024 10:05
@sylvlecl sylvlecl marked this pull request as ready for review September 3, 2024 10:09
Signed-off-by: Sylvain Leclerc <[email protected]>
Signed-off-by: Sylvain Leclerc <[email protected]>
Copy link
Contributor

@mabw-rte mabw-rte Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have an idea why this was not corrected by isort ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually there is no change on import sort order in the final PR. Intermediate commits introduced unwanted changes (related to another PR).

Copy link
Contributor

@mabw-rte mabw-rte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sylvlecl

  • For the rebase it should be launched locally (always).
  • The commit messages should be corrected.
  • The branch naming does not respect the git flow we fixed.

@sylvlecl
Copy link
Member Author

sylvlecl commented Sep 4, 2024

Closed in favor of #2133 which fixes naming issues

@sylvlecl sylvlecl closed this Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants