-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve aggregate test results function #813
Conversation
This PR includes changes to |
Codecov ReportAll modified and coverable lines are covered by tests ✅ ✅ All tests successful. No failed tests found. 📢 Thoughts on this report? Let us know! |
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## main #813 +/- ##
=======================================
Coverage 96.25% 96.26%
=======================================
Files 814 814
Lines 18659 18689 +30
=======================================
+ Hits 17961 17991 +30
Misses 698 698
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
@@ -893,14 +892,15 @@ def test_test_results_no_tests(self) -> None: | |||
def test_branch_filter_on_test_results(self) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
am I missing something or are there no assertions on this test and the next two?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think the github file viewer is just not showing them, they are there
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh weird, yeah I see them now
requirements.in
Outdated
@@ -45,7 +45,7 @@ pytz | |||
redis | |||
regex | |||
requests | |||
sentry-sdk>=2.13.0 | |||
sentry-sdk>=1.40.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this intended? :O
class Array(Aggregate): | ||
function = "array" | ||
|
||
|
||
def aggregate_test_results( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reminder to update the doc for this fn
utils/test_results.py
Outdated
When(outcome__in=["failure", "error"], then=Value(1.0)), | ||
totals = DailyTestRollup.objects.filter(repoid=repoid, date__gt=time_ago) | ||
|
||
print([(t.pass_count, t.fail_count) for t in totals], branch) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still want this? :P
utils/test_results.py
Outdated
) | ||
|
||
return failure_rates_queryset | ||
print(aggregation_of_test_results.query) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still need this? :P
|
||
aggregation_of_test_results = totals.values("test").annotate( | ||
failure_rate=( | ||
Cast(Sum(F("fail_count")), output_field=FloatField()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does the Cast(Sum(F("fail_count"))) get cached when we do this calc twice? I don't have an idea of how we'd simplify, mostly curious if you knew
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i would expect postgres to take care of this in a performant way
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
this commit also adds the LastDuration field to the TestResult GQL model. The reason we're using latest_run and not the updated_at field of the object is because in the future we want to parse the timestamp from the JUnit XML file so latest_run will be tied to that and not the time at which we process the test results.
5f3decc
to
b8a56cc
Compare
❌ 16 Tests Failed:
View the top 3 failed tests by shortest run time
To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard |
Test Failures Detected: Due to failing tests, we cannot provide coverage reports at this time. ❌ Failed Test Results:Completed 2329 tests with View the full list of failed testspytest
|
This PR does 2 things:
depends on: codecov/worker#699