Let flaky adaptive tests be rerun #9043
Conversation
These adaptive localization tests fail very often on weak hardware, and is hampering the deploy process. Adding reruns on failure will lead to similar hampering when the test actually starts to fail for the correct reasons, but this we have to live with.
|
I think we would have to check whether this works, by rerunning the actions until we observe a retry. Currently the tests timeout at 600s. If the failure mechanism (seems to be some sort of connection problem) would propagate to a failure immediately rather than a deadlock, then that would also help. |
There was a problem hiding this comment.
I think if we can show that the test is retried or that this makes the workflow more stable then we can merge the PR. Mind, #8787 should be high priority as clearly something is really wrong on mac.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9043 +/- ##
==========================================
+ Coverage 90.81% 90.84% +0.03%
==========================================
Files 352 352
Lines 21746 21746
==========================================
+ Hits 19749 19756 +7
+ Misses 1997 1990 -7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
These adaptive localization tests fail very often on weak hardware, and is hampering the deploy process.
Adding reruns on failure will lead to similar hampering when the test actually starts to fail for the correct reasons, but this we have to live with.
Issue
Resolves pain by 🩹 (#8787)
Approach
♻️
git rebase -i main --exec 'pytest tests/ert/unit_tests -n logical -m "not integration_test"')When applicable