Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up regtests by reorganizing test_nirspec_ifu_spec3 #8811

Merged
merged 2 commits into from
Sep 26, 2024

Conversation

braingram
Copy link
Collaborator

@braingram braingram commented Sep 20, 2024

This is a minor reorganization of tests in test_nirspec_ifu_spec3 to more efficiently run them in parallel.

This reduces regtests runtime by about 8 minutes.

jwst/regtest/test_nirspec_ifu_spec3.py contains 2 of the slower regtests since the regtests run in parallel scoped by module these 2 tests end up in the same scope given to 1 worker. This worker then spends over 2 hours on those 2 tests (run serially). This PR splits jwst/regtest/test_nirspec_ifu_spec3.py putting the 2 slow fixtures in different files/modules so they can be run on different workers.

Ideally we'd start the slowest tests first so they could toil away while other workers run other tests. If we instead start them later in the test run our overall run will likely end up taking as we wait for the slow test to finish. However this is currently not possible with pytest-xdist (see pytest-dev/pytest-xdist#778).

With this PR regtests ran in 1hr 8 minutes: https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1745/ (with only unrelated failures).

Tasks

  • request a review from someone specific, to avoid making the maintainers review every PR
  • add a build milestone, i.e. Build 11.3 (use the latest build if not sure)
  • Does this PR change user-facing code / API?
    • add an entry to CHANGES.rst within the relevant release section (otherwise add the no-changelog-entry-needed label to this PR)
    • update or add relevant tests
    • update relevant docstrings and / or docs/ page
    • start a regression test and include a link to the running job (click here for instructions)
      • Do truth files need to be updated ("okified")?
        • after the reviewer has approved these changes, run okify_regtests to update the truth files
  • if a JIRA ticket exists, make sure it is resolved properly

Copy link

codecov bot commented Sep 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.86%. Comparing base (eab0532) to head (e860360).
Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8811   +/-   ##
=======================================
  Coverage   61.86%   61.86%           
=======================================
  Files         377      377           
  Lines       38911    38911           
=======================================
  Hits        24071    24071           
  Misses      14840    14840           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@braingram braingram marked this pull request as ready for review September 20, 2024 18:35
@braingram braingram requested a review from a team as a code owner September 20, 2024 18:35
@braingram braingram added this to the Build 11.2 milestone Sep 20, 2024
@braingram braingram changed the title WIP: Regtest org Split test_nirspec_ifu_spec3 into 2 files to allow parallel running of slow tests Sep 20, 2024
@braingram braingram changed the title Split test_nirspec_ifu_spec3 into 2 files to allow parallel running of slow tests Speed up regtests by reorganizing test_nirspec_ifu_spec3 Sep 24, 2024
Copy link
Collaborator

@melanieclarke melanieclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! I've asked @jemorrison to take a quick look too, since she added the pixel replace test.

Copy link
Collaborator

@jemorrison jemorrison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok to me

@braingram braingram merged commit 976c239 into spacetelescope:main Sep 26, 2024
30 checks passed
@braingram braingram deleted the regtest_org branch September 26, 2024 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants