[Security Solution] Support testing with different roles in Serverless Cypress and API integration tests #164451
Comments
banderror
added
test
Team:Detections and Resp
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Hi @maximpn / @banderror , FYI: https://github.com/elastic/security-team/issues/7614 We currently have this service here that uses the cc/ @gergoabraham , @kevinlog |
|
Thanks for the info @paul-tavares, much appreciated! |
|
Also - Found this link yesterday that seems to have instructions/process on how ot add roles/users to ES in serverless. I'm sure our process (role/user loader) will likely need to be adjusted use the SAML approach: I believe this is currently being used for the And actually - I just looked at it and it seems to have our security roles in it. I wonder if they are already loaded 🤔 . I'm currently trying to get Fleet-server running in serverless CI/locally so I can't spend time on this just yet, but I figure I would pass along the info. cc/ @kevinlog , @gergoabraham , @tomsonpl ☝️ 😱 |
|
The problem has been solved by @paul-tavares for the Defend Workflows Cypress tests in #167446. TL;DR:
Note that the source file for Serverless Security roles is located at https://github.com/elastic/project-controller/blob/69ed9829dc53bedac0f92781070dcc64073c79b4/internal/project/security/config/roles.yml. Its counterpart in the Kibana repo is In the future, we will need to automate syncing these files and ideally consolidate them in a single place. It would be a separate ticket to work on. cc @maximpn |
|
Please note that all the above is for the first quality gate, things may be a little bit different on the second one since we are talking about a deployed environment on MKI. |
|
@MadameSheema Sure thing. I'd imagine that on MKI roles would be built-in, and we'd need to create users having these roles and add them to the project under test using Cloud APIs. That's out of the scope of this ticket, I'll mention this in the description. |
banderror
changed the title
|
@maximpn Could you please also address this issue for API integration tests in the new |
|
@banderror sure, anyway we have tightly coupled roles related code in functional and Cypress tests so it's logical to handle both of them. |
…oles (#169017) **Addresses:** #164451 ## Summary This PR allows to run role based reused between ESS and Serverless Cypress tests. ## Details The main idea behind is to make environmental differences for tests unnoticeable. As Serverless env already has roles and users but ESS env allows to create any possible role and user we just need to create Serverless roles and corresponding users + specific ESS roles and corresponding users in ESS env before running any ESS tests. This way tests will run in a similar env and don't have to bother by roles/users creation in test suites. This is achieved by using separate Cypress support files (Cypress includes `support/e2e.js` by default) `ess_e2e.ts` and `serverless_e2e.ts` executed for corresponding environments. `ess_e2e.ts` contains logic to create mentioned above roles and users while `serverless_e2e.ts` doesn't contain such logic. _Only one user created per role and user has the same name as its corresponding role with `changeme` password._ To have an ability to create roles we need to store their definitions somewhere. It's also convenient to have JSON definitions instead of YAML. Plus Serverless roles should be pulled from `project-controller` repo but it's not addressed in this PR. I've chosen the following locations - Serverless Security roles in `packages/kbn-es/src/serverless_resources/security_roles.json`. While `@kbn/es` is a common package it has `serverless_resources` folder containing `roles.yml` with a mix of `https://github.com/elastic/project-controller/blob/main/internal/project/observability/config/roles.yml`, `https://github.com/elastic/project-controller/blob/main/internal/project/esproject/config/roles.yml` and `https://github.com/elastic/project-controller/blob/main/internal/project/security/config/roles.yml` copied from `project-controller` and used for ES data restore. As there is no automation yet it looks logical to keep Security roles subset next to ES Serverless resources. - ESS Security specific roles in `x-pack/plugins/security_solution/common/test/ess_roles.json` On top of that the following has been done - `reader` role replaced with `t1_analyst` where possible in tests (besides `e2e/explore/cases/attach_alert_to_case.cy.ts` but it's purely ESS test so it's fine) as `reader` is ESS specific and make harder to run the same tests in ESS and Serverless environments but both roles are almost equivalent - `login()` helper function accepts all known roles (Serverless + ESS) but throws an exception if a custom ESS role is used under Serverless env - `x-pack/plugins/security_solution/server/lib/detection_engine/scripts/roles_users` isn't necessary anymore as `security_roles.json` + `ess_roles.json` contain all the necessary data to create roles and users ### Does it enable role support for MKI environments? No. This PR only enabling role support for Non-MKI Serverless environments. MKI env has predefined roles but not users. This will be addressed in a follow up PR. ## Flaky test runner Two unskiped in this PR Serverless Cypress tests using non default role `detection_response/detection_alerts/missing_privileges_callout.cy.ts` and `detection_response/prebuilt_rules/prebuilt_rules_install_update_authorization.cy.ts` [150 runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3723) 🟢 (there is one env related failure but it doesn't look related to the changes in this PR)
…oles (elastic#169017) **Addresses:** elastic#164451 ## Summary This PR allows to run role based reused between ESS and Serverless Cypress tests. ## Details The main idea behind is to make environmental differences for tests unnoticeable. As Serverless env already has roles and users but ESS env allows to create any possible role and user we just need to create Serverless roles and corresponding users + specific ESS roles and corresponding users in ESS env before running any ESS tests. This way tests will run in a similar env and don't have to bother by roles/users creation in test suites. This is achieved by using separate Cypress support files (Cypress includes `support/e2e.js` by default) `ess_e2e.ts` and `serverless_e2e.ts` executed for corresponding environments. `ess_e2e.ts` contains logic to create mentioned above roles and users while `serverless_e2e.ts` doesn't contain such logic. _Only one user created per role and user has the same name as its corresponding role with `changeme` password._ To have an ability to create roles we need to store their definitions somewhere. It's also convenient to have JSON definitions instead of YAML. Plus Serverless roles should be pulled from `project-controller` repo but it's not addressed in this PR. I've chosen the following locations - Serverless Security roles in `packages/kbn-es/src/serverless_resources/security_roles.json`. While `@kbn/es` is a common package it has `serverless_resources` folder containing `roles.yml` with a mix of `https://github.com/elastic/project-controller/blob/main/internal/project/observability/config/roles.yml`, `https://github.com/elastic/project-controller/blob/main/internal/project/esproject/config/roles.yml` and `https://github.com/elastic/project-controller/blob/main/internal/project/security/config/roles.yml` copied from `project-controller` and used for ES data restore. As there is no automation yet it looks logical to keep Security roles subset next to ES Serverless resources. - ESS Security specific roles in `x-pack/plugins/security_solution/common/test/ess_roles.json` On top of that the following has been done - `reader` role replaced with `t1_analyst` where possible in tests (besides `e2e/explore/cases/attach_alert_to_case.cy.ts` but it's purely ESS test so it's fine) as `reader` is ESS specific and make harder to run the same tests in ESS and Serverless environments but both roles are almost equivalent - `login()` helper function accepts all known roles (Serverless + ESS) but throws an exception if a custom ESS role is used under Serverless env - `x-pack/plugins/security_solution/server/lib/detection_engine/scripts/roles_users` isn't necessary anymore as `security_roles.json` + `ess_roles.json` contain all the necessary data to create roles and users ### Does it enable role support for MKI environments? No. This PR only enabling role support for Non-MKI Serverless environments. MKI env has predefined roles but not users. This will be addressed in a follow up PR. ## Flaky test runner Two unskiped in this PR Serverless Cypress tests using non default role `detection_response/detection_alerts/missing_privileges_callout.cy.ts` and `detection_response/prebuilt_rules/prebuilt_rules_install_update_authorization.cy.ts` [150 runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3723) 🟢 (there is one env related failure but it doesn't look related to the changes in this PR)
|
Just a quick update on progress here after syncing with @maximpn. After merging #169017, at the time of writing this comment we are in the following state:
We can run Cypress tests under different roles in both ESS and Serverless locally and in CI for PRs. Support for roles in MKI pipelines (QA, Staging, and Production quality gates) is not there yet, as well as support for roles in any API integration tests in Serverless. Merging #170131 would bring support for roles in API integration tests locally and in CI for PRs:
|
yctercero
added
Test:Cypress
Team:Detection Engine
|
Pinging @elastic/security-detection-engine (Team:Detection Engine) |
|
@banderror currently for serverless you can use any prebuilt role for Cypress and API. Custom roles for API are ready as well. For Cypress we have the following ticket to implement it. |
|
UPDATE:
|
Epic: #153633
Summary
User roles in Serverless environments will be predefined: users won't be able to create custom roles, and thus we won't be able to create them in our tests either.
We need to find a way to test features on behalf of a user that will have one of the predefined roles. For Cypress tests, testing using a role should be transparent and should just work in both ESS and Serverless. The way of setting up users and roles should be abstracted away via Cypress commands and/or tasks, and the concrete implementation of these commands should be different in the Serverless and ESS Cypress configs.
NOTE: This ticket is concerned only about running Cypress tests locally and in the PR pipeline (the so-called first quality gate). Running tests under different roles against a real Serverless environment in Cloud is out of the scope.
The text was updated successfully, but these errors were encountered: