Failing test: Detection Engine API Integration Tests - ESS - Rule Execution Logic.x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_execution_logic/execution_logic/esql·ts - Rule execution logic API Execution logic @ess ES|QL rule type aggregating query rules should aggregate results correctly #171967
Assignees
Labels
failed-test
A test failure on a tracked branch, potentially flaky-test
Team:Detection Engine
Security Solution Detection Engine Area
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
triage_needed
Comments
kibanamachine
added
the
failed-test
banderror
added
triage_needed
Team:Detections and Resp
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
vitaliidm
added a commit
that referenced
this issue
Mar 4, 2024
…177226) ## Summary Adressess: - #175147 - #171967 - #174555 All of failed tests happened only once throughout all the time they were executed, which hints its flakiness probability is very low. However, I made few small changes I think would help to avoid this potential flakiness. For FTR tests, I adde sort order when retrieving preview alerts for their subsequent assertions. It looks like, flaky failures happened because order of returned alerts wasn't as expected in the tests. For Cypress test, I added additional check if input is not disabled before typing new values in it. Looks like, flaky failure happened because of input was not enabled on the time when new value should've been typed in it. Flaky test runner: FTR: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5232 Cypress: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5231
|
fixed in #177226 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Mar 4, 2024
…lastic#177226) ## Summary Adressess: - elastic#175147 - elastic#171967 - elastic#174555 All of failed tests happened only once throughout all the time they were executed, which hints its flakiness probability is very low. However, I made few small changes I think would help to avoid this potential flakiness. For FTR tests, I adde sort order when retrieving preview alerts for their subsequent assertions. It looks like, flaky failures happened because order of returned alerts wasn't as expected in the tests. For Cypress test, I added additional check if input is not disabled before typing new values in it. Looks like, flaky failure happened because of input was not enabled on the time when new value should've been typed in it. Flaky test runner: FTR: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5232 Cypress: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5231 (cherry picked from commit 0af37ed)
kibanamachine
referenced
this issue
Mar 4, 2024
…tests (#177226) (#177968) # Backport This will backport the following commits from `main` to `8.13`: - [[Security Solution][Detection Engine] fixes potentially flaky tests (#177226)](#177226) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Vitalii Dmyterko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-03-04T18:14:32Z","message":"[Security Solution][Detection Engine] fixes potentially flaky tests (#177226)\n\n## Summary\r\n\r\nAdressess:\r\n - https://github.com/elastic/kibana/issues/175147\r\n - https://github.com/elastic/kibana/issues/171967\r\n - https://github.com/elastic/kibana/issues/174555\r\n\r\nAll of failed tests happened only once throughout all the time they were\r\nexecuted, which hints its flakiness probability is very low.\r\n\r\nHowever, I made few small changes I think would help to avoid this\r\npotential flakiness.\r\n\r\nFor FTR tests, I adde sort order when retrieving preview alerts for\r\ntheir subsequent assertions. It looks like, flaky failures happened\r\nbecause order of returned alerts wasn't as expected in the tests.\r\n\r\nFor Cypress test, I added additional check if input is not disabled\r\nbefore typing new values in it. Looks like, flaky failure happened\r\nbecause of input was not enabled on the time when new value should've\r\nbeen typed in it.\r\n\r\n\r\n\r\nFlaky test runner:\r\n\r\nFTR:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5232\r\nCypress:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5231","sha":"0af37ed2aec88dc7fd8abc328892a14aff8ecc8e","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team: SecuritySolution","backport:prev-minor","Team:Detection Engine","8.13 candidate","v8.14.0"],"title":"[Security Solution][Detection Engine] fixes potentially flaky tests","number":177226,"url":"https://github.com/elastic/kibana/pull/177226","mergeCommit":{"message":"[Security Solution][Detection Engine] fixes potentially flaky tests (#177226)\n\n## Summary\r\n\r\nAdressess:\r\n - https://github.com/elastic/kibana/issues/175147\r\n - https://github.com/elastic/kibana/issues/171967\r\n - https://github.com/elastic/kibana/issues/174555\r\n\r\nAll of failed tests happened only once throughout all the time they were\r\nexecuted, which hints its flakiness probability is very low.\r\n\r\nHowever, I made few small changes I think would help to avoid this\r\npotential flakiness.\r\n\r\nFor FTR tests, I adde sort order when retrieving preview alerts for\r\ntheir subsequent assertions. It looks like, flaky failures happened\r\nbecause order of returned alerts wasn't as expected in the tests.\r\n\r\nFor Cypress test, I added additional check if input is not disabled\r\nbefore typing new values in it. Looks like, flaky failure happened\r\nbecause of input was not enabled on the time when new value should've\r\nbeen typed in it.\r\n\r\n\r\n\r\nFlaky test runner:\r\n\r\nFTR:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5232\r\nCypress:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5231","sha":"0af37ed2aec88dc7fd8abc328892a14aff8ecc8e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/177226","number":177226,"mergeCommit":{"message":"[Security Solution][Detection Engine] fixes potentially flaky tests (#177226)\n\n## Summary\r\n\r\nAdressess:\r\n - https://github.com/elastic/kibana/issues/175147\r\n - https://github.com/elastic/kibana/issues/171967\r\n - https://github.com/elastic/kibana/issues/174555\r\n\r\nAll of failed tests happened only once throughout all the time they were\r\nexecuted, which hints its flakiness probability is very low.\r\n\r\nHowever, I made few small changes I think would help to avoid this\r\npotential flakiness.\r\n\r\nFor FTR tests, I adde sort order when retrieving preview alerts for\r\ntheir subsequent assertions. It looks like, flaky failures happened\r\nbecause order of returned alerts wasn't as expected in the tests.\r\n\r\nFor Cypress test, I added additional check if input is not disabled\r\nbefore typing new values in it. Looks like, flaky failure happened\r\nbecause of input was not enabled on the time when new value should've\r\nbeen typed in it.\r\n\r\n\r\n\r\nFlaky test runner:\r\n\r\nFTR:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5232\r\nCypress:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5231","sha":"0af37ed2aec88dc7fd8abc328892a14aff8ecc8e"}}]}] BACKPORT--> Co-authored-by: Vitalii Dmyterko <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
A test failed on a tracked branch
First failure: CI Build - main
The text was updated successfully, but these errors were encountered: