Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Security Solution Cypress.x-pack/plugins/security_solution/public/management/cypress/e2e/response_actions/alerts_response_console·cy·ts - Response console From Alerts "before all" hook for "should open responder from alert details flyout" "before all" hook for "should open responder from alert details flyout" #169689

Closed
kibanamachine opened this issue Oct 24, 2023 · 25 comments · Fixed by #174341
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test OLM Sprint Team:Defend Workflows “EDR Workflows” sub-team of Security Solution

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Oct 24, 2023

A test failed on a tracked branch

CypressError: `cy.task('createEndpointHost')` timed out after waiting `900000ms`.

https://on.cypress.io/api/task

Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Response console`

Although you have test retries enabled, we do not retry tests when `before all` or `after all` hooks fail
    at <unknown> (http://localhost:5639/__cypress/runner/cypress_runner.js:135256:70)
    at tryCatcher (http://localhost:5639/__cypress/runner/cypress_runner.js:1807:23)
    at <unknown> (http://localhost:5639/__cypress/runner/cypress_runner.js:4186:41)
    at tryCatcher (http://localhost:5639/__cypress/runner/cypress_runner.js:1807:23)
    at Promise._settlePromiseFromHandler (http://localhost:5639/__cypress/runner/cypress_runner.js:1519:31)
    at Promise._settlePromise (http://localhost:5639/__cypress/runner/cypress_runner.js:1576:18)
    at Promise._settlePromise0 (http://localhost:5639/__cypress/runner/cypress_runner.js:1621:10)
    at Promise._settlePromises (http://localhost:5639/__cypress/runner/cypress_runner.js:1697:18)
    at _drainQueueStep (http://localhost:5639/__cypress/runner/cypress_runner.js:2407:12)
    at _drainQueue (http://localhost:5639/__cypress/runner/cypress_runner.js:2400:9)
    at Async._drainQueues (http://localhost:5639/__cypress/runner/cypress_runner.js:2416:5)
    at Async.drainQueues (http://localhost:5639/__cypress/runner/cypress_runner.js:2286:14)
From Your Spec Code:
    at createEndpointHost (webpack:///./tasks/create_endpoint_host.ts:15:12)
    at Context.eval (webpack:///./e2e/response_actions/alerts_response_console.cy.ts:46:36)

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 24, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 24, 2023
@kibanamachine kibanamachine added the Team:Defend Workflows “EDR Workflows” sub-team of Security Solution label Oct 24, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 24, 2023
szwarckonrad added a commit that referenced this issue Oct 25, 2023
…169092)

Restart vagrant vm on error during `beforeAll` task `createEndpointHost`

Defend Workflows Cypress suite ran 300 times through flaky test runner:
1. 100x
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699
2. 50x
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3707
3. 50x
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3708
4. 50x
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3709
5. 50x
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3710


Flaky test runner runs with `createEndpointHost` task failure with
successful recovery:
1.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3710#018b62fd-9ae9-4988-b1e0-ab0f04d8efdc
2.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3710#018b62fd-9ae6-4340-992b-1474ee0f114b
3.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3708#018b62fd-578e-4817-ae1c-8c58e8774eec
4.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3708#018b62fd-5787-4245-85a6-cb446e42bc73
5.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3707#018b62fc-fc17-407e-88de-d0b43b6b1d44
(failed due to unrelated issue)
6.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2c3-430c-b3e3-72b9fbb22d24
7.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2c6-4315-b828-b3218a70f209
8.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2c7-4ff7-9a70-7354f90179e0
9.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2d7-418f-b043-049e5effb26f
10.
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2da-47cc-b4ea-a4d4de3ba0a0

New errors not spotted before that got to do with env set up:

1. `vagrant up` failed:
1.1
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3708#018b62fd-5787-4245-85a6-cb446e42bc73
1.2
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2d0-4a52-87d9-34caa8927465

2. `CypressError: `cy.task('indexFleetEndpointPolicy')` timed out after
waiting `60000ms`.:
2.1
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3707#018b62fc-fc04-40d4-b155-46f094681edb
2.2
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2c9-4ebb-9174-eb9d79d04d02
2.3
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3699#018b61d9-d2dc-438f-94b0-9f94ae95701c
    

Closes:
#168284
#169343
#169468
#169469
#169467
#169465
#169466
#169157
#168719
#168427
#168359
#168340
#169689

---------

Co-authored-by: Patryk Kopyciński <[email protected]>
@kibanamachine kibanamachine reopened this Nov 6, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kevinlog
Copy link
Contributor

kevinlog commented Nov 6, 2023

lates failure:

1) "before all" hook for "should open responder from alert details flyout"
--
  | proc [kibana] [2023-11-06T16:49:22.304+00:00][WARN ][plugins.fleet] Setting tamper protection for Agent Policy 29d0788a-a682-4aa8-943f-78426264be4b to false {"service":{"node":{"roles":["background_tasks","ui"]}}}
  |  
  |  
  | 0 passing (1m)
  | 1 failing
  |  
  | 1) Response console
  | From Alerts
  | "before all" hook for "should open responder from alert details flyout":
  | CypressError: `cy.task('createEndpointHost')` failed with the following error:
  |  
  | > Failed to retrieve list of versions from elastic's artifact repository: Service Temporarily Unavailable (HTTP 503) {URL: https://artifacts-api.elastic.co/v1/versions)
  |  
  | https://on.cypress.io/api/task
  |  
  | Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Response console`
  |  
  | Although you have test retries enabled, we do not retry tests when `before all` or `after all` hooks fail

Same issue with other tests running into "Service Temporarily unavailable"

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic
Copy link
Member

mistic commented Nov 8, 2023

Skipped.

main: 7d1eaab

ashokaditya added a commit to ashokaditya/kibana that referenced this issue Nov 29, 2023
@ashokaditya
Copy link
Member

PR /pull/172152

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic
Copy link
Member

mistic commented Dec 19, 2023

Skipped.

main: 9d2e0a6

@dasansol92 dasansol92 removed their assignment Jan 4, 2024
ashokaditya added a commit to ashokaditya/kibana that referenced this issue Jan 5, 2024
ashokaditya added a commit that referenced this issue Jan 5, 2024
…74341)

## Summary

Re-enable response console tests for Alerts page.

closes /issues/169689
closes /issues/173472

### Flaky test runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4766
x 100 (all pass)

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
delanni pushed a commit to delanni/kibana that referenced this issue Jan 11, 2024
…astic#174341)

## Summary

Re-enable response console tests for Alerts page.

closes elastic/issues/169689
closes elastic/issues/173472

### Flaky test runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4766
x 100 (all pass)

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
@kibanamachine kibanamachine reopened this Jan 26, 2024
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.12

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.12

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.12

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.12

@mistic
Copy link
Member

mistic commented Jan 27, 2024

/skip

@kibanamachine
Copy link
Contributor Author

Skipped

8.12: 4c61892

dasansol92 added a commit that referenced this issue Feb 2, 2024
## Summary
Re-enable skipped e2e tests due vagrant up error. This pr depends on
this change: #175755
Originally merged at main here:
#175754

fixes: #170811
fixes: #170674
fixes: #169343
fixes: #170601
fixes: #173464
fixes: #170667
fixes: #170812
fixes: #170604
fixes: #170373
fixes: #169958
fixes: #170814
fixes: #170424
fixes: #170706
fixes: #170563
fixes: #169689
fixes: #170817
fixes: #169821
fixes: #170816
fixes: #170794

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: Kibana Machine <[email protected]>
@dasansol92
Copy link
Contributor

#175780

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test OLM Sprint Team:Defend Workflows “EDR Workflows” sub-team of Security Solution
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants