Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] fix endpoint list + metadata api FTR tests #170489

Merged
merged 2 commits into from
Dec 11, 2023
Merged

[Security Solution] fix endpoint list + metadata api FTR tests #170489

merged 2 commits into from
Dec 11, 2023

Conversation

joeypoon
Copy link
Member

@joeypoon joeypoon commented Nov 3, 2023 •
edited by kibanamachine
Loading

Summary

Fix endpoint list + metadata api FTR tests. Caused by .fleet-agents index occasionally not returning docs for united metadata transform which is resolved by the additional search on the .fleet-agents index. Still investigating root cause of why this is happening and why the search solves flakiness (replacing the search with a refresh does not resolve flakiness).

Flaky test runs:

Fixes:

Checklist

For maintainers

@joeypoon joeypoon force-pushed the fix/flaky-tests branch 6 times, most recently from 4e058b4 to 31733f1 Compare November 16, 2023 21:59
@joeypoon joeypoon force-pushed the fix/flaky-tests branch 2 times, most recently from b989d75 to dfbf6bf Compare December 1, 2023 00:27
@joeypoon joeypoon changed the title [Security Solution] fix flaky endpoint list ftr test [Security Solution] unskip endpoint list FTR tests Dec 1, 2023
@joeypoon joeypoon force-pushed the fix/flaky-tests branch 20 times, most recently from 1732180 to 411de6c Compare December 6, 2023 03:10
@joeypoon joeypoon requested a review from a team as a code owner December 10, 2023 22:38
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

joeypoon
joeypoon commented Dec 10, 2023
View reviewed changes
x-pack/plugins/fleet/server/services/epm/elasticsearch/retry.ts
const isRetryableError = (e: any, additionalResponseStatuses: number[] = []) =>
retryMessages.some((msg) => e.message.includes(msg)) ||
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no_shard_available_action_exception encompasses multiple status codes and sometimes can be 500 which is a bit too wide to include. just checking the message allows for a more narrow check.

@botelastic botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Dec 10, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

joeypoon
joeypoon commented Dec 10, 2023
View reviewed changes
x-pack/plugins/security_solution/common/endpoint/utils/package_v2.ts
return semver.includes('-') ? semver.substring(0, semver.indexOf('-')) : semver;
}

const MIN_ENDPOINT_PACKAGE_V2_VERSION = '8.13.0';
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Endpoint package spec/unattended upgrade has been bumped from 8.12.x

joeypoon
joeypoon commented Dec 10, 2023
View reviewed changes
x-pack/plugins/security_solution/common/endpoint/utils/package_v2.ts
export function isEndpointPackageV2(version: string) {
return semverLte(MIN_ENDPOINT_PACKAGE_V2_VERSION, version);
const parsedVersion = parseSemver(version);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We weren't catching preview versions previously meaning all preview versions were returning false.

juliaElastic
juliaElastic approved these changes Dec 11, 2023
View reviewed changes
Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fleet change LGTM

@joeypoon joeypoon enabled auto-merge (squash) December 11, 2023 15:47
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

  • 💛 Build #182333 was flaky f05562696a6856f61d4a5628bc57bd9c7b607ea1
  • 💔 Build #182051 failed 411de6cbf1a334f5d19c04cdb3f6a66373dabd88
  • 💔 Build #182048 failed 17321802c299f987d776b64a3f49a7cace0d6e57
  • 💔 Build #182046 failed a6950243a3b71f89417c6110f82a5c7184e17699
  • 💔 Build #182045 failed 82c366231b0655412da174aba1c8dcd6ff7bd789
  • 💔 Build #182016 failed c008b7974aeef8d34a7823635adb0f750d8ab59a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@joeypoon joeypoon merged commit a50b97f into elastic:main Dec 11, 2023
38 checks passed
@joeypoon joeypoon deleted the fix/flaky-tests branch December 11, 2023 17:05
@kibanamachine kibanamachine mentioned this pull request Dec 11, 2023
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 11, 2023
@joeypoon
[Security Solution] fix endpoint list + metadata api FTR tests (elast…
40eef6b 
…ic#170489)

(cherry picked from commit a50b97f)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.12

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Dec 11, 2023
@kibanamachine @joeypoon
[8.12] [Security Solution] fix endpoint list + metadata api FTR tests (…
5935014 
…#170489) (#173085)

# Backport

This will backport the following commits from `main` to `8.12`:
- [[Security Solution] fix endpoint list + metadata api FTR tests
(#170489)](#170489)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Joey F.
Poon","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-12-11T16:59:53Z","message":"[Security
Solution] fix endpoint list + metadata api FTR tests
(#170489)","sha":"a50b97fb2382213c88f65bc7317caa792c3c0750","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","Team:Defend
Workflows","v8.12.0","v8.13.0"],"number":170489,"url":"https://github.com/elastic/kibana/pull/170489","mergeCommit":{"message":"[Security
Solution] fix endpoint list + metadata api FTR tests
(#170489)","sha":"a50b97fb2382213c88f65bc7317caa792c3c0750"}},"sourceBranch":"main","suggestedTargetBranches":["8.12"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.13.0","labelRegex":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/170489","number":170489,"mergeCommit":{"message":"[Security
Solution] fix endpoint list + metadata api FTR tests
(#170489)","sha":"a50b97fb2382213c88f65bc7317caa792c3c0750"}}]}]
BACKPORT-->

Co-authored-by: Joey F. Poon <[email protected]>
@ashokaditya ashokaditya mentioned this pull request Jan 24, 2024
Merged
1 task
ashokaditya added a commit that referenced this pull request Feb 23, 2024
@ashokaditya
[Security Solution][Endpoint] Remove responder FTR test (#175454)
db8440d 
## Summary

Delete responder test skipped in /pull/170489.
We've cypress tests that cover the test cases here.

### Flaky runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937
x 50 (...)

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 23, 2024
@ashokaditya
[Security Solution][Endpoint] Remove responder FTR test (elastic#175454)
e17a735 
## Summary

Delete responder test skipped in elastic/pull/170489.
We've cypress tests that cover the test cases here.

### Flaky runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937
x 50 (...)

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed

(cherry picked from commit db8440d)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 23, 2024
@ashokaditya
[Security Solution][Endpoint] Remove responder FTR test (elastic#175454)
86be10f 
## Summary

Delete responder test skipped in elastic/pull/170489.
We've cypress tests that cover the test cases here.

### Flaky runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937
x 50 (...)

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed

(cherry picked from commit db8440d)
kibanamachine added a commit that referenced this pull request Feb 23, 2024
@kibanamachine @ashokaditya
[8.13] [Security Solution][Endpoint] Remove responder FTR test (#175454…
0a089f4 
…) (#177740)

# Backport

This will backport the following commits from `main` to `8.13`:
- [[Security Solution][Endpoint] Remove responder FTR test
(#175454)](#175454)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT
[{"author":{"name":"Ash","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-02-23T17:02:47Z","message":"[Security
Solution][Endpoint] Remove responder FTR test (#175454)\n\n##
Summary\r\n\r\nDelete responder test skipped in
/pull/170489.\r\nWe've cypress tests that cover the test
cases here.\r\n\r\n### Flaky
runner\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937\r\nx
50 (...)\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"db8440dc5b0d56bb147b6c629d9ea229ccf96fcf","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","OLM
Sprint","v8.12.1","v8.13.0","v8.14.0"],"title":"[Security
Solution][Endpoint] Remove responder FTR
test","number":175454,"url":"https://github.com/elastic/kibana/pull/175454","mergeCommit":{"message":"[Security
Solution][Endpoint] Remove responder FTR test (#175454)\n\n##
Summary\r\n\r\nDelete responder test skipped in
/pull/170489.\r\nWe've cypress tests that cover the test
cases here.\r\n\r\n### Flaky
runner\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937\r\nx
50 (...)\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"db8440dc5b0d56bb147b6c629d9ea229ccf96fcf"}},"sourceBranch":"main","suggestedTargetBranches":["8.12","8.13"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/175454","number":175454,"mergeCommit":{"message":"[Security
Solution][Endpoint] Remove responder FTR test (#175454)\n\n##
Summary\r\n\r\nDelete responder test skipped in
/pull/170489.\r\nWe've cypress tests that cover the test
cases here.\r\n\r\n### Flaky
runner\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937\r\nx
50 (...)\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"db8440dc5b0d56bb147b6c629d9ea229ccf96fcf"}}]}]
BACKPORT-->

Co-authored-by: Ash <[email protected]>
kibanamachine added a commit that referenced this pull request Feb 23, 2024
@kibanamachine @ashokaditya
[8.12] [Security Solution][Endpoint] Remove responder FTR test (#175454…
4f3971a 
…) (#177739)

# Backport

This will backport the following commits from `main` to `8.12`:
- [[Security Solution][Endpoint] Remove responder FTR test
(#175454)](#175454)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT
[{"author":{"name":"Ash","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-02-23T17:02:47Z","message":"[Security
Solution][Endpoint] Remove responder FTR test (#175454)\n\n##
Summary\r\n\r\nDelete responder test skipped in
/pull/170489.\r\nWe've cypress tests that cover the test
cases here.\r\n\r\n### Flaky
runner\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937\r\nx
50 (...)\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"db8440dc5b0d56bb147b6c629d9ea229ccf96fcf","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","OLM
Sprint","v8.12.1","v8.13.0","v8.14.0"],"title":"[Security
Solution][Endpoint] Remove responder FTR
test","number":175454,"url":"https://github.com/elastic/kibana/pull/175454","mergeCommit":{"message":"[Security
Solution][Endpoint] Remove responder FTR test (#175454)\n\n##
Summary\r\n\r\nDelete responder test skipped in
/pull/170489.\r\nWe've cypress tests that cover the test
cases here.\r\n\r\n### Flaky
runner\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937\r\nx
50 (...)\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"db8440dc5b0d56bb147b6c629d9ea229ccf96fcf"}},"sourceBranch":"main","suggestedTargetBranches":["8.12","8.13"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/175454","number":175454,"mergeCommit":{"message":"[Security
Solution][Endpoint] Remove responder FTR test (#175454)\n\n##
Summary\r\n\r\nDelete responder test skipped in
/pull/170489.\r\nWe've cypress tests that cover the test
cases here.\r\n\r\n### Flaky
runner\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937\r\nx
50 (...)\r\n\r\n### Checklist\r\n\r\n- [x] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests
changed","sha":"db8440dc5b0d56bb147b6c629d9ea229ccf96fcf"}}]}]
BACKPORT-->

Co-authored-by: Ash <[email protected]>
semd pushed a commit to semd/kibana that referenced this pull request Mar 4, 2024
@ashokaditya @semd
[Security Solution][Endpoint] Remove responder FTR test (elastic#175454)
ba82001 
## Summary

Delete responder test skipped in elastic/pull/170489.
We've cypress tests that cover the test cases here.

### Flaky runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937
x 50 (...)

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
fkanout pushed a commit to fkanout/kibana that referenced this pull request Mar 4, 2024
@ashokaditya @fkanout
[Security Solution][Endpoint] Remove responder FTR test (elastic#175454)
1b3d766 
## Summary

Delete responder test skipped in elastic/pull/170489.
We've cypress tests that cover the test cases here.

### Flaky runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4937
x 50 (...)

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juliaElastic juliaElastic juliaElastic approved these changes

@szwarckonrad szwarckonrad szwarckonrad approved these changes

@ashokaditya ashokaditya ashokaditya approved these changes

Assignees
No one assigned
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team:Fleet Team label for Observability Data Collection Fleet team v8.12.0 v8.13.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@joeypoon @elasticmachine @kibana-ci @kibanamachine @ashokaditya @szwarckonrad @juliaElastic