Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Security Solution Cypress.x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts/alert_status·cy·ts - Changing alert status Opening alerts can bulk open alerts can bulk open alerts #169091

Open
kibanamachine opened this issue Oct 17, 2023 · 11 comments
Labels
consider-next failed-test A test failure on a tracked branch, potentially flaky-test Feature:Detection Alerts Security Solution Detection Alerts Feature legit-flake Test was triaged and marked as an actual flake. skipped-test Team:Detection Engine Security Solution Detection Engine Area Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.13.0

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Oct 17, 2023

A test failed on a tracked branch

AssertionError: Timed out retrying after 150000ms: Expected to find content: '4' within the element: <span.css-ldwgxs> but never did.
    at Context.eval (webpack:///./e2e/detection_response/detection_alerts/alert_status.cy.ts:107:35)

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 17, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 17, 2023
@kibanamachine kibanamachine added the Team:Detection Engine Security Solution Detection Engine Area label Oct 17, 2023
@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 17, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic
Copy link
Member

mistic commented Oct 18, 2023

Skipped.

main: 4f2d1db

@jpdjere jpdjere added the legit-flake Test was triaged and marked as an actual flake. label Oct 19, 2023
@jpdjere
Copy link
Contributor

jpdjere commented Oct 19, 2023

maximpn pushed a commit that referenced this issue Oct 24, 2023
…ts and enable Serverless run (#169218)

## Summary

Flaky test runner for Cypress tests in:
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts` 
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists` 
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation` 

## Changes

- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`  **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts` **added disableAutorefresh() to avoid flakiness**

## Related failing-test issues

### `detection_response/detection_alerts`
1. #169091
**Marked as `legit-flake` and assigned to @elastic/security-detection-engine** 
3. #163885

### `detection_response/value_lists`
4. #165699

### `detection_response/rule_creation`
5. #163691

## Flaky test runner

- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615 [SERVERLESS ONLY]
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656 [Serverless without indicator match and saved queries]
jpdjere added a commit to jpdjere/kibana that referenced this issue Oct 24, 2023
…ts and enable Serverless run (elastic#169218)

## Summary

Flaky test runner for Cypress tests in:
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`

## Changes

- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts` **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`  **tagged to run on Serverless**
- `x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts` **added disableAutorefresh() to avoid flakiness**

## Related failing-test issues

### `detection_response/detection_alerts`
1. elastic#169091
**Marked as `legit-flake` and assigned to @elastic/security-detection-engine**
3. elastic#163885

### `detection_response/value_lists`
4. elastic#165699

### `detection_response/rule_creation`
5. elastic#163691

## Flaky test runner

- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615 [SERVERLESS ONLY]
- https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656 [Serverless without indicator match and saved queries]

(cherry picked from commit 939c87d)
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

jpdjere referenced this issue Oct 26, 2023
…ess tests and enable Serverless run (#169218) (#169681)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Unskip tests for `detection_response` Cypress
tests and enable Serverless run
(#169218)](#169218)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-24T07:18:42Z","message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","Team:Detections
and Resp","Team:Detection Rule Management","Team:Detection
Engine","v8.12.0"],"number":169218,"url":"https://github.com/elastic/kibana/pull/169218","mergeCommit":{"message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169218","number":169218,"mergeCommit":{"message":"[Security
Solution] Unskip tests for `detection_response` Cypress tests and enable
Serverless run (#169218)\n\n## Summary\r\n\r\nFlaky test runner for
Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_alerts`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists`
\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation`
\r\n\r\n## Changes\r\n\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule_data_view.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/event_correlation_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/machine_learning_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/new_terms_rule.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/override.cy.ts`
**tagged to run on Serverless**\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/deletion/rule_delete.cy.ts`
**added disableAutorefresh() to avoid flakiness**\r\n\r\n## Related
failing-test issues\r\n\r\n###
`detection_response/detection_alerts`\r\n1.
https://github.com/elastic/kibana/issues/169091\r\n**Marked as
`legit-flake` and assigned to @elastic/security-detection-engine**
\r\n3. https://github.com/elastic/kibana/issues/163885\r\n\r\n###
`detection_response/value_lists`\r\n4.
https://github.com/elastic/kibana/issues/165699\r\n\r\n###
`detection_response/rule_creation`\r\n5.
https://github.com/elastic/kibana/issues/163691\r\n\r\n## Flaky test
runner\r\n\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3583\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3615
[SERVERLESS ONLY]\r\n-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3656
[Serverless without indicator match and saved
queries]","sha":"939c87d95342c61f03c853cb3b34c9f412c303b0"}}]}]
BACKPORT-->

Co-authored-by: Kibana Machine <[email protected]>
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@jpdjere
Copy link
Contributor

jpdjere commented Nov 7, 2023

Skipping all tests as part of PR: #170540

Flake resurfaced and blocked pipeline here: #170663 (comment)

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

mistic added a commit that referenced this issue Nov 16, 2023
@banderror banderror added Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Dec 15, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@yctercero yctercero removed the blocker label Mar 13, 2024
@yctercero yctercero removed their assignment Mar 20, 2024
@yctercero yctercero added Test:Cypress Team:Detection Engine Security Solution Detection Engine Area and removed Team:Detection Engine Security Solution Detection Engine Area Test:Cypress labels Dec 27, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-engine (Team:Detection Engine)

@yctercero yctercero added the Feature:Detection Alerts Security Solution Detection Alerts Feature label Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
consider-next failed-test A test failure on a tracked branch, potentially flaky-test Feature:Detection Alerts Security Solution Detection Alerts Feature legit-flake Test was triaged and marked as an actual flake. skipped-test Team:Detection Engine Security Solution Detection Engine Area Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.13.0
Projects
None yet
Development

No branches or pull requests

7 participants