Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger flaky test runner via GitHub #72591

Closed
timroes opened this issue Jul 21, 2020 · 3 comments
Closed

Trigger flaky test runner via GitHub #72591

timroes opened this issue Jul 21, 2020 · 3 comments
Labels
Feature:Development impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:Operations Team label for Operations Team Team:QA Team label for QA Team

Comments

@timroes
Copy link
Contributor

timroes commented Jul 21, 2020

It would be very nice if we could trigger the flaky test runner from within a PR via a command/bot directly.

My ideal scenario, would just be a comment: @elasticmachine flaky-test-runner and it automatically determines which test groups have been changed within the PR and trigger a run for that and leave a comment afterwards with the result.

Since I think it might be trigger figuring this out automatically (also what would you run when you just change a page object or service?), already being able to specify it manually would be a big win: @elasticmachine flaky-test-runner oss:ciGroup:5 and then get a comment under it.

What are the benefits why we should build this:

  • Currently flaky test runs are kind of not documented under a PR properly. Sometimes users leave comments and link to the flaky test runs they triggered (which then will later on just vanish). Having a bot comment under the PR makes this significantly more transparent and saves it for posterity. Also currently you need to check yourself when the flaky test runner is done, a comment would trigger your regular GitHub notifications.
  • We have external people working on Kibana, without an Elastic account (aka most of our visualization team). They currently cannot trigger that, since they can't log into the CI. That bot would allow every Contributor to use the flaky test runner, without requiring someone to trigger their test runs for them.

Related to #70733

@timroes timroes added Team:Operations Team label for Operations Team Team:QA Team label for QA Team Feature:Development labels Jul 21, 2020
@timroes
Copy link
Contributor Author

timroes commented Jul 21, 2020

cc @alexwizp

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-qa (Team:QA)

@tylersmalley tylersmalley added 1 and removed 1 labels Oct 11, 2021
@exalate-issue-sync exalate-issue-sync bot added impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:small Small Level of Effort labels Feb 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Development impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:Operations Team label for Operations Team Team:QA Team label for QA Team
Projects
None yet
Development

No branches or pull requests

3 participants