Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Chrome X-Pack UI Functional Tests - ML short_tests.x-pack/test/functional/apps/ml/short_tests/model_management/model_list·ts - machine learning - short tests model management trained models for ML power user with imported models deletes the imported model pt_tiny_pass_through #175443

Closed
kibanamachine opened this issue Jan 24, 2024 · 15 comments · Fixed by #177407 or #192851
Closed

Failing test: Chrome X-Pack UI Functional Tests - ML short_tests.x-pack/test/functional/apps/ml/short_tests/model_management/model_list·ts - machine learning - short tests model management trained models for ML power user with imported models deletes the imported model pt_tiny_pass_through #175443

kibanamachine opened this issue Jan 24, 2024 · 15 comments · Fixed by #177407 or #192851
Assignees
@darnautov
Labels
failed-test A test failure on a tracked branch, potentially flaky-test :ml

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Jan 24, 2024 •
edited
Loading

A test failed on a tracked branch

Error: retry.tryForTime timeout: Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj~="mlModelsTable"] [data-test-subj~="row-pt_tiny_pass_through"] [data-test-subj="euiCollapsedItemActionsButton"])
Wait timed out after 10053ms
    at /var/lib/buildkite-agent/builds/kb-n2-4-spot-262f2164bd719cf6/elastic/kibana-on-merge/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:17
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at onFailure (retry_for_success.ts:17:9)
    at retryForSuccess (retry_for_success.ts:67:13)
    at RetryService.try (retry.ts:37:12)
    at Proxy.clickByCssSelector (find.ts:419:5)
    at TestSubjects.click (test_subjects.ts:164:5)
    at common_ui.ts:428:11
    at runAttempt (retry_for_success.ts:29:15)
    at retryForSuccess (retry_for_success.ts:79:21)
    at RetryService.tryForTime (retry.ts:23:12)
    at Object.invokeTableRowAction (common_ui.ts:424:7)
    at ModelsTable.deleteModel (trained_models_table.ts:335:7)
    at Context.<anonymous> (model_list.ts:496:13)
    at Object.apply (wrap_function.js:73:16)
    at onFailure (retry_for_success.ts:17:9)
    at retryForSuccess (retry_for_success.ts:67:13)
    at RetryService.tryForTime (retry.ts:23:12)
    at Object.invokeTableRowAction (common_ui.ts:424:7)
    at ModelsTable.deleteModel (trained_models_table.ts:335:7)
    at Context.<anonymous> (model_list.ts:496:13)
    at Object.apply (wrap_function.js:73:16)

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Jan 24, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Jan 24, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Jan 24, 2024
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

mistic added a commit that referenced this issue Jan 25, 2024
@mistic
skip flaky suite (#175443)
3f02a72
@mistic
Copy link
Member

mistic commented Jan 25, 2024

Skipped.

main: 3f02a72

jloleysens added a commit that referenced this issue Jan 25, 2024
@jloleysens
Merge branch 'main' into versioned-router-and-oas
f5151f6 
* main: (520 commits)
  Update Kibana code editor dependencies (#171720)
  [SLOs] Hide view in app in slo alerts table in slo details page (#175441)
  [api-docs] 2024-01-25 Daily api_docs build (#175502)
  [DOCS] Add buildkite links to doc preview comments (#175463)
  skip flaky suite (#175443)
  [Security Solution][Timeline] refactor timeline modal save timeline button (#175343)
  [RAM] Stack Management::Rules loses user selections when navigating back (#174954)
  [Security Solution][Timeline] refactor timeline modal attach to case button (#175163)
  Upgrade EUI to v92.1.1 (#174955)
  [Fleet]: Beta label is shown inconsistently while selecting proxy under Fleet settings. (#170634)
  [Cloud Security] Rules Combo Box filters Custom component (#175175)
  skip flaky suite (#175407)
  [Security Solution][Timeline] refactor timeline modal open timeline button (#175335)
  [Embedded Console] Introduce kbnSolutionNavOffset CSS variable (#175348)
  [Console] disable access to embedded console without dev tools capability (#175321)
  fix(x-pack/reporting): use FIPS-compliant ID generator `uuidv4` in Reporting plugin (#174809)
  [Security Solution] Data quality dashboard persistence (#173185)
  [RAM][Observability] Add alert fields table to Observability flyout (#174685)
  test: add missing await for connector table disappearance (#175430)
  [RAM][Maintenance Window] Fix maintenance window FE types and transforms  (#173888)
  ...
@peteharverson
Copy link
Contributor

Removing blocker label as the functionality (deleting an imported third party model) works as expected in manual testing.

Failure is on the check that the collapsed actions menu button euiCollapsedItemActionsButton exists for the row.

@darnautov darnautov self-assigned this Jan 29, 2024
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this issue Feb 15, 2024
@mistic @CoenWarmer
skip flaky suite (elastic#175443)
012098e
@darnautov darnautov mentioned this issue Feb 21, 2024
Merged
2 tasks
darnautov added a commit that referenced this issue Feb 22, 2024
@darnautov
[ML] Enable function tests for imported trained models (#177407)
84889d0 
## Summary

Closes #175443

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Feb 22, 2024
@darnautov
[ML] Enable function tests for imported trained models (elastic#177407)
79fcbb9 
## Summary

Closes elastic#175443

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed

(cherry picked from commit 84889d0)
kibanamachine referenced this issue Feb 22, 2024
@kibanamachine @darnautov
[8.13] [ML] Enable function tests for imported trained models (#177407)…
f733bdb 
… (#177567)

# Backport

This will backport the following commits from `main` to `8.13`:
- [[ML] Enable function tests for imported trained models
(#177407)](#177407)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dima
Arnautov","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-02-22T10:54:35Z","message":"[ML]
Enable function tests for imported trained models (#177407)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/175443\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"84889d0399b222c9372dd9602a044ae5ff9b12c2","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":[":ml","test_ui_functional","release_note:skip","Team:ML","v8.13.0","v8.14.0"],"title":"[ML]
Enable function tests for imported trained models
","number":177407,"url":"https://github.com/elastic/kibana/pull/177407","mergeCommit":{"message":"[ML]
Enable function tests for imported trained models (#177407)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/175443\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"84889d0399b222c9372dd9602a044ae5ff9b12c2"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/177407","number":177407,"mergeCommit":{"message":"[ML]
Enable function tests for imported trained models (#177407)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/175443\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"84889d0399b222c9372dd9602a044ae5ff9b12c2"}}]}]
BACKPORT-->

Co-authored-by: Dima Arnautov <[email protected]>
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine kibanamachine reopened this Feb 23, 2024
fkanout pushed a commit to fkanout/kibana that referenced this issue Mar 4, 2024
@darnautov @fkanout
[ML] Enable function tests for imported trained models (elastic#177407)
74b4f05 
## Summary

Closes elastic#175443

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

mistic added a commit that referenced this issue Jul 4, 2024
@mistic
skip flaky suite (#175443)
ed038ad
@mistic
Copy link
Member

mistic commented Jul 4, 2024

Skipped.

main: ed038ad

@peteharverson
Copy link
Contributor

Removing the blocker label. Models can be deleted successfully in the trained models UI. Will look into a fix for the flaky test for the actions menu items.

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@kibanamachine
Copy link
Contributor Author

Skipped

8.15: dd891cd

kibanamachine added a commit that referenced this issue Jul 25, 2024
@kibanamachine
skip failing test suite (#175443)
dd891cd
@mistic mistic removed the blocker label Jul 25, 2024
@pheyos pheyos mentioned this issue Sep 13, 2024
Merged
@pheyos pheyos closed this as completed in c1b7d82 Sep 13, 2024
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Sep 13, 2024
@pheyos
[ML] Functional tests - stabilize model list tests (elastic#192851)
7261e21 
## Summary

This PR stabilizes the model list tests by adding a wait for global
loading before checking the collapsed action button.

### Details

I was able to reproduce the original failure by running the test suite
in my local browser with network speed throttled to 3G: after stopping a
model deployment, the row actions change and during that time the global
loading is active. Checking the collapsed action button too early (i.e.
before global loading finished) returns a false positive and then the
next step will try to click that button which in the mean time
disappeared. Adding the wait for global loading before checking the
collapsed actions button fixed for my local runs with throttling.

Closes elastic#175443

(cherry picked from commit c1b7d82)
kibanamachine added a commit that referenced this issue Sep 13, 2024
@kibanamachine @pheyos
[8.x] [ML] Functional tests - stabilize model list tests (#192851) (#…
6d066c2 
…192869)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ML] Functional tests - stabilize model list tests
(#192851)](#192851)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Robert
Oskamp","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-13T13:37:30Z","message":"[ML]
Functional tests - stabilize model list tests (#192851)\n\n##
Summary\r\n\r\nThis PR stabilizes the model list tests by adding a wait
for global\r\nloading before checking the collapsed action
button.\r\n\r\n### Details\r\n\r\nI was able to reproduce the original
failure by running the test suite\r\nin my local browser with network
speed throttled to 3G: after stopping a\r\nmodel deployment, the row
actions change and during that time the global\r\nloading is active.
Checking the collapsed action button too early (i.e.\r\nbefore global
loading finished) returns a false positive and then the\r\nnext step
will try to click that button which in the mean time\r\ndisappeared.
Adding the wait for global loading before checking the\r\ncollapsed
actions button fixed for my local runs with throttling.\r\n\r\nCloses
#175443","sha":"c1b7d8254e571cc15d6a2c44d8b4ec0b3c4a11b8","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":[":ml","release_note:skip","v9.0.0","backport:prev-minor","v8.16.0"],"title":"[ML]
Functional tests - stabilize model list
tests","number":192851,"url":"https://github.com/elastic/kibana/pull/192851","mergeCommit":{"message":"[ML]
Functional tests - stabilize model list tests (#192851)\n\n##
Summary\r\n\r\nThis PR stabilizes the model list tests by adding a wait
for global\r\nloading before checking the collapsed action
button.\r\n\r\n### Details\r\n\r\nI was able to reproduce the original
failure by running the test suite\r\nin my local browser with network
speed throttled to 3G: after stopping a\r\nmodel deployment, the row
actions change and during that time the global\r\nloading is active.
Checking the collapsed action button too early (i.e.\r\nbefore global
loading finished) returns a false positive and then the\r\nnext step
will try to click that button which in the mean time\r\ndisappeared.
Adding the wait for global loading before checking the\r\ncollapsed
actions button fixed for my local runs with throttling.\r\n\r\nCloses
#175443","sha":"c1b7d8254e571cc15d6a2c44d8b4ec0b3c4a11b8"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192851","number":192851,"mergeCommit":{"message":"[ML]
Functional tests - stabilize model list tests (#192851)\n\n##
Summary\r\n\r\nThis PR stabilizes the model list tests by adding a wait
for global\r\nloading before checking the collapsed action
button.\r\n\r\n### Details\r\n\r\nI was able to reproduce the original
failure by running the test suite\r\nin my local browser with network
speed throttled to 3G: after stopping a\r\nmodel deployment, the row
actions change and during that time the global\r\nloading is active.
Checking the collapsed action button too early (i.e.\r\nbefore global
loading finished) returns a false positive and then the\r\nnext step
will try to click that button which in the mean time\r\ndisappeared.
Adding the wait for global loading before checking the\r\ncollapsed
actions button fixed for my local runs with throttling.\r\n\r\nCloses
#175443","sha":"c1b7d8254e571cc15d6a2c44d8b4ec0b3c4a11b8"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Robert Oskamp <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@darnautov darnautov

Labels
failed-test A test failure on a tracked branch, potentially flaky-test :ml
Projects
None yet
Milestone
No milestone
5 participants
@mistic @darnautov @peteharverson @elasticmachine @kibanamachine