[Detection Engine][FTR] Add FTR exception list tests for Serverless prebuilt roles #198420
Conversation
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7308[✅] x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/authentication/exceptions/common/trial_license_complete_tier/configs/serverless.config.ts: 25/25 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7310[✅] x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/authentication/exceptions/common/trial_license_complete_tier/configs/serverless.config.ts: 25/25 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7311[✅] x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/authentication/exceptions/common/trial_license_complete_tier/configs/serverless.config.ts: 25/25 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7316[❌] x-pack/test/security_solution_api_integration/test_suites/lists_and_exception_lists/authentication/exceptions/common/trial_license_complete_tier/configs/serverless.config.ts: 7/25 tests passed. |
All passed on rebuild. |
yctercero
added
release_note:skip
|
Pinging @elastic/security-detection-engine (Team:Detection Engine) |
There was a problem hiding this comment.
Thank you for the tests! Other than my questions about the configuration and directory mismatch, I have one other small nit. I think the authentication/exceptions directory should be renamed to authorization/exceptions. We're not really authenticating roles, more so we are checking if a user (role) is authorized for the specific operations for exceptions. Looks great!
| * 2.0. | ||
| */ | ||
|
|
||
| import { createTestConfig } from '../../../../../../../config/serverless/config.base.essentials'; |
There was a problem hiding this comment.
This config is listed under a directory trial_license_complete_tier but it is using the serverless essentials configuration. I think one of these things is incorrect?
There was a problem hiding this comment.
Thanks! I fixed it up!
| * 2.0. | ||
| */ | ||
|
|
||
| import { createTestConfig } from '../../../../../../../config/serverless/config.base.essentials'; |
There was a problem hiding this comment.
same question as above - essentials config but under the complete tier directory.
…a into ftr_rbac_exception_tests
|
@dhurley14 thanks for the feedback! I've updated the folder structure to make more sense and renamed to |
💚 Build Succeeded
Metrics [docs]
History
cc @yctercero |
|
Starting backport for target branches: 8.x |
…rebuilt roles (elastic#198420) ## Summary Adds FTR tests that check our Serverless prebuilt roles against our exception list endpoints. We have had little coverage or visibility to know if any changes made in elasticsearch-controller introduce a bug in our prebuilt roles. We could certainly discuss how such tests should be organized - I chose to create an `authentication` folder that then has a matching folder for the other sections and a file for each prebuilt role. With us nearing GA, I'd like to prioritize having coverage and following up with any improvements. (cherry picked from commit 4c649d9)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…less prebuilt roles (#198420) (#199192) # Backport This will backport the following commits from `main` to `8.x`: - [[Detection Engine][FTR] Add FTR exception list tests for Serverless prebuilt roles (#198420)](#198420) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Yara Tercero","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-06T17:38:01Z","message":"[Detection Engine][FTR] Add FTR exception list tests for Serverless prebuilt roles (#198420)\n\n## Summary\r\n\r\nAdds FTR tests that check our Serverless prebuilt roles against our\r\nexception list endpoints.\r\n\r\nWe have had little coverage or visibility to know if any changes made in\r\nelasticsearch-controller introduce a bug in our prebuilt roles.\r\n\r\nWe could certainly discuss how such tests should be organized - I chose\r\nto create an `authentication` folder that then has a matching folder for\r\nthe other sections and a file for each prebuilt role. With us nearing\r\nGA, I'd like to prioritize having coverage and following up with any\r\nimprovements.","sha":"4c649d9f14d6631d23931fc0d32dc28017cbad38","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Detection Engine","backport:version","v8.17.0"],"title":"[Detection Engine][FTR] Add FTR exception list tests for Serverless prebuilt roles","number":198420,"url":"https://github.com/elastic/kibana/pull/198420","mergeCommit":{"message":"[Detection Engine][FTR] Add FTR exception list tests for Serverless prebuilt roles (#198420)\n\n## Summary\r\n\r\nAdds FTR tests that check our Serverless prebuilt roles against our\r\nexception list endpoints.\r\n\r\nWe have had little coverage or visibility to know if any changes made in\r\nelasticsearch-controller introduce a bug in our prebuilt roles.\r\n\r\nWe could certainly discuss how such tests should be organized - I chose\r\nto create an `authentication` folder that then has a matching folder for\r\nthe other sections and a file for each prebuilt role. With us nearing\r\nGA, I'd like to prioritize having coverage and following up with any\r\nimprovements.","sha":"4c649d9f14d6631d23931fc0d32dc28017cbad38"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/198420","number":198420,"mergeCommit":{"message":"[Detection Engine][FTR] Add FTR exception list tests for Serverless prebuilt roles (#198420)\n\n## Summary\r\n\r\nAdds FTR tests that check our Serverless prebuilt roles against our\r\nexception list endpoints.\r\n\r\nWe have had little coverage or visibility to know if any changes made in\r\nelasticsearch-controller introduce a bug in our prebuilt roles.\r\n\r\nWe could certainly discuss how such tests should be organized - I chose\r\nto create an `authentication` folder that then has a matching folder for\r\nthe other sections and a file for each prebuilt role. With us nearing\r\nGA, I'd like to prioritize having coverage and following up with any\r\nimprovements.","sha":"4c649d9f14d6631d23931fc0d32dc28017cbad38"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Yara Tercero <[email protected]>
…rebuilt roles (elastic#198420) ## Summary Adds FTR tests that check our Serverless prebuilt roles against our exception list endpoints. We have had little coverage or visibility to know if any changes made in elasticsearch-controller introduce a bug in our prebuilt roles. We could certainly discuss how such tests should be organized - I chose to create an `authentication` folder that then has a matching folder for the other sections and a file for each prebuilt role. With us nearing GA, I'd like to prioritize having coverage and following up with any improvements.
Summary
Adds FTR tests that check our Serverless prebuilt roles against our exception list endpoints.
We have had little coverage or visibility to know if any changes made in elasticsearch-controller introduce a bug in our prebuilt roles.
We could certainly discuss how such tests should be organized - I chose to create an
authenticationfolder that then has a matching folder for the other sections and a file for each prebuilt role. With us nearing GA, I'd like to prioritize having coverage and following up with any improvements.