Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test runner should post result to the associated PR #173129

Closed
sorenlouv opened this issue Dec 12, 2023 · 2 comments · Fixed by #183043
Closed

Flaky test runner should post result to the associated PR #173129

sorenlouv opened this issue Dec 12, 2023 · 2 comments · Fixed by #183043
Assignees
@delanni
Labels
Team:Operations Team label for Operations Team

Comments

@sorenlouv
Copy link
Member

sorenlouv commented Dec 12, 2023 •
edited
Loading

It would be great if the flaky test runner posted the results of the run to the associated PR. This way the author can start the flaky test runner, and automatically be notified when it finishes (via github comment notification). Also, it makes the result visible to others like reviewers who will be interested in the stability of newly introduced tests.

Example

✅ Flaky tests runner: 50/50 passed (build)

or

❌ Flaky tests runner: 39/50 passed (build)
@botelastic botelastic bot added the needs-team Issues missing a team label label Dec 12, 2023
@sorenlouv sorenlouv added Team:Operations Team label for Operations Team and removed needs-team Issues missing a team label labels Dec 12, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@delanni delanni self-assigned this May 8, 2024
@delanni delanni mentioned this issue May 9, 2024
Merged
@delanni
Copy link
Contributor

delanni commented May 10, 2024

WIP:
#183043 (comment)

delanni added a commit that referenced this issue May 13, 2024
@delanni
[CI] Comment flaky test results on tested PR (#183043)
38d4230 
## Summary
Extends the flaky-test-runner with the capability to comment on the
flaky test runs on the PR that's being tested.

Closes: #173129

- chore(flaky-test-runner): Add a step to collect results and comment on
the tested PR
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue May 13, 2024
@delanni
[CI] Comment flaky test results on tested PR (elastic#183043)
93677b4 
## Summary
Extends the flaky-test-runner with the capability to comment on the
flaky test runs on the PR that's being tested.

Closes: elastic#173129

- chore(flaky-test-runner): Add a step to collect results and comment on
the tested PR

(cherry picked from commit 38d4230)
kibanamachine referenced this issue May 13, 2024
@kibanamachine @delanni
[8.14] [CI] Comment flaky test results on tested PR (#183043) (#183261)
2efb339 
# Backport

This will backport the following commits from `main` to `8.14`:
- [[CI] Comment flaky test results on tested PR
(#183043)](#183043)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Alex
Szabo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-05-13T10:47:30Z","message":"[CI]
Comment flaky test results on tested PR (#183043)\n\n##
Summary\r\nExtends the flaky-test-runner with the capability to comment
on the\r\nflaky test runs on the PR that's being tested.\r\n\r\nCloses:
https://github.com/elastic/kibana/issues/173129\r\n\r\n-
chore(flaky-test-runner): Add a step to collect results and comment
on\r\nthe tested
PR","sha":"38d4230e617171c429850c0cff2ef22860204976","branchLabelMapping":{"^v8.15.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:all-open","v8.15.0"],"title":"[CI]
Comment flaky test results on tested
PR","number":183043,"url":"https://github.com/elastic/kibana/pull/183043","mergeCommit":{"message":"[CI]
Comment flaky test results on tested PR (#183043)\n\n##
Summary\r\nExtends the flaky-test-runner with the capability to comment
on the\r\nflaky test runs on the PR that's being tested.\r\n\r\nCloses:
https://github.com/elastic/kibana/issues/173129\r\n\r\n-
chore(flaky-test-runner): Add a step to collect results and comment
on\r\nthe tested
PR","sha":"38d4230e617171c429850c0cff2ef22860204976"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.15.0","branchLabelMappingKey":"^v8.15.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/183043","number":183043,"mergeCommit":{"message":"[CI]
Comment flaky test results on tested PR (#183043)\n\n##
Summary\r\nExtends the flaky-test-runner with the capability to comment
on the\r\nflaky test runs on the PR that's being tested.\r\n\r\nCloses:
https://github.com/elastic/kibana/issues/173129\r\n\r\n-
chore(flaky-test-runner): Add a step to collect results and comment
on\r\nthe tested
PR","sha":"38d4230e617171c429850c0cff2ef22860204976"}}]}] BACKPORT-->

Co-authored-by: Alex Szabo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@delanni delanni

Labels
Team:Operations Team label for Operations Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[CI] Comment flaky test results on tested PR delanni/kibana
3 participants
@sorenlouv @delanni @elasticmachine