Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Chrome X-Pack UI Functional Tests - ML anomaly_detection_result_views.x-pack/test/functional/apps/ml/anomaly_detection_result_views/forecasts·ts - machine learning - anomaly detection forecasts with single metric job displays job results #164381

Closed
kibanamachine opened this issue Aug 21, 2023 · 24 comments · Fixed by #167820 or #192737
Closed

Failing test: Chrome X-Pack UI Functional Tests - ML anomaly_detection_result_views.x-pack/test/functional/apps/ml/anomaly_detection_result_views/forecasts·ts - machine learning - anomaly detection forecasts with single metric job displays job results #164381

kibanamachine opened this issue Aug 21, 2023 · 24 comments · Fixed by #167820 or #192737
Assignees
@pheyos
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Feature:Anomaly Detection ML anomaly detection :ml

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Aug 21, 2023 •
edited
Loading

A test failed on a tracked branch

Error: expected testSubject(mlForecastValuesline) to exist
    at TestSubjects.existOrFail (test_subjects.ts:71:13)
    at Object.assertForecastChartElementsExists (forecast.ts:38:7)
    at Context.<anonymous> (forecasts.ts:101:9)
    at Object.apply (wrap_function.js:73:16)

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Aug 21, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Aug 21, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Aug 21, 2023
@mistic mistic closed this as completed Aug 22, 2023
@kibanamachine kibanamachine reopened this Aug 22, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

mistic added a commit that referenced this issue Aug 22, 2023
@mistic
skip flaky suite (#164381)
83a0eec
@mistic
Copy link
Member

mistic commented Aug 22, 2023

Skipped.

main: 83a0eec

@peteharverson
Copy link
Contributor

Removing the blocker label as the functionality is working correctly in manual tests. The functional test is failing because the time range slider is not moving after running the forecast to display the forecasted time range.

image

There must be some flakiness in the way the slider moves to show the forecast time range.

@peteharverson peteharverson added Feature:Anomaly Detection ML anomaly detection and removed blocker labels Aug 25, 2023
@alvarezmelissa87 alvarezmelissa87 mentioned this issue Oct 2, 2023
Merged
2 tasks
alvarezmelissa87 added a commit that referenced this issue Oct 3, 2023
@alvarezmelissa87
[ML] Anomaly detection results functional tests: retry forecast chart…
b3b9472 
… check (#167820)

## Summary

Fixes #164381
Flaky test runner build:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3279

This PR ensures enough time is given for the forecast chart to show up
correctly by adding retries.


### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.14

@kibanamachine kibanamachine reopened this Jun 12, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.14

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@delanni
Copy link
Contributor

delanni commented Jun 12, 2024 •
edited
Loading

/skip

Skipping it because it started to fail over all 8.x branches, and PRs: 8.13, 8.14, main

@peteharverson
Copy link
Contributor

Closing - has been unskipped in #187795. Looks like recent failures were caused by something in the testing environment - likely the chrome version.

@szaffarano szaffarano mentioned this issue Jul 12, 2024
Closed
@kibanamachine kibanamachine reopened this Aug 2, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

mistic added a commit that referenced this issue Aug 6, 2024
@mistic
skip flaky suite (#164381)
b6eee03
@mistic
Copy link
Member

mistic commented Aug 6, 2024

Skipped.

main: b6eee03

@alvarezmelissa87
Copy link
Contributor

Removing the blocker label as the functionality is working correctly in manual testing

@pheyos pheyos mentioned this issue Sep 13, 2024
Merged
@pheyos pheyos closed this as completed in e528c8b Sep 13, 2024
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Sep 13, 2024
@pheyos
[ML] Functional tests - stabilize forecast tests (elastic#192737)
25779b7 
## Summary

This PR stabilizes and re-enables the forecast tests in the ML
permission and results view test suites by waiting for global loading to
finish before checking the forecast `Run` button state.

### Details

I was able to reproduce the original failures by running the test suites
in my local browser with network speed throttled to `3G`: the forecast
modal opened and the `Run` button was disabled during global page
loading, so the enabled check failed. Introducing the global loading
wait fixed the tests for the throttled local runs.

Closes elastic#189565
Closes elastic#189537
Closes elastic#164381

(cherry picked from commit e528c8b)
kibanamachine added a commit that referenced this issue Sep 13, 2024
@kibanamachine @pheyos
[8.x] [ML] Functional tests - stabilize forecast tests (#192737) (#19…
ce00858 
…2861)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ML] Functional tests - stabilize forecast tests
(#192737)](#192737)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Robert
Oskamp","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-13T12:48:51Z","message":"[ML]
Functional tests - stabilize forecast tests (#192737)\n\n##
Summary\r\n\r\nThis PR stabilizes and re-enables the forecast tests in
the ML\r\npermission and results view test suites by waiting for global
loading to\r\nfinish before checking the forecast `Run` button
state.\r\n\r\n### Details\r\n\r\nI was able to reproduce the original
failures by running the test suites\r\nin my local browser with network
speed throttled to `3G`: the forecast\r\nmodal opened and the `Run`
button was disabled during global page\r\nloading, so the enabled check
failed. Introducing the global loading\r\nwait fixed the tests for the
throttled local runs.\r\n\r\nCloses #189565\r\nCloses #189537\r\nCloses
#164381","sha":"e528c8bd8e49ec928df8f08a219d7b4df79c81cb","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":[":ml","Feature:Anomaly
Detection","release_note:skip","v9.0.0","backport:prev-minor","v8.16.0"],"title":"[ML]
Functional tests - stabilize forecast
tests","number":192737,"url":"https://github.com/elastic/kibana/pull/192737","mergeCommit":{"message":"[ML]
Functional tests - stabilize forecast tests (#192737)\n\n##
Summary\r\n\r\nThis PR stabilizes and re-enables the forecast tests in
the ML\r\npermission and results view test suites by waiting for global
loading to\r\nfinish before checking the forecast `Run` button
state.\r\n\r\n### Details\r\n\r\nI was able to reproduce the original
failures by running the test suites\r\nin my local browser with network
speed throttled to `3G`: the forecast\r\nmodal opened and the `Run`
button was disabled during global page\r\nloading, so the enabled check
failed. Introducing the global loading\r\nwait fixed the tests for the
throttled local runs.\r\n\r\nCloses #189565\r\nCloses #189537\r\nCloses
#164381","sha":"e528c8bd8e49ec928df8f08a219d7b4df79c81cb"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192737","number":192737,"mergeCommit":{"message":"[ML]
Functional tests - stabilize forecast tests (#192737)\n\n##
Summary\r\n\r\nThis PR stabilizes and re-enables the forecast tests in
the ML\r\npermission and results view test suites by waiting for global
loading to\r\nfinish before checking the forecast `Run` button
state.\r\n\r\n### Details\r\n\r\nI was able to reproduce the original
failures by running the test suites\r\nin my local browser with network
speed throttled to `3G`: the forecast\r\nmodal opened and the `Run`
button was disabled during global page\r\nloading, so the enabled check
failed. Introducing the global loading\r\nwait fixed the tests for the
throttled local runs.\r\n\r\nCloses #189565\r\nCloses #189537\r\nCloses
#164381","sha":"e528c8bd8e49ec928df8f08a219d7b4df79c81cb"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Robert Oskamp <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pheyos pheyos

Labels
failed-test A test failure on a tracked branch, potentially flaky-test Feature:Anomaly Detection ML anomaly detection :ml
Projects
None yet
Milestone
No milestone
7 participants
@pheyos @mistic @delanni @alvarezmelissa87 @peteharverson @elasticmachine @kibanamachine