Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[APM] Migrate /test/apm_api_integration/tests/inspect/ to be deployment-agnostic API tests #198977

Closed
rmyz opened this issue Nov 5, 2024 · 3 comments · Fixed by #199963
Closed

[APM] Migrate /test/apm_api_integration/tests/inspect/ to be deployment-agnostic API tests #198977

rmyz opened this issue Nov 5, 2024 · 3 comments · Fixed by #199963
Assignees
@iblancof
Labels
Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team technical debt Improvement of the software architecture and operational architecture

Comments

@rmyz
Copy link
Contributor

rmyz commented Nov 5, 2024 •
edited by iblancof
Loading

API ESS Serverless Test files
GET /internal/apm/environments ✅ ❌ ESS: ./test/apm_api_integration/tests/inspect/inspect.spec.ts
@rmyz rmyz added Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team technical debt Improvement of the software architecture and operational architecture labels Nov 5, 2024
@iblancof iblancof self-assigned this Nov 12, 2024
@iblancof
Copy link
Contributor

The APM inspect test suite relies on Agent Configuration, which, based on my checks, isn’t available in Serverless. Therefore, we can’t migrate this to be deployment-agnostic.

1)    Serverless Observability - Deployment-agnostic api integration tests
   │       APM
   │         inspect
   │           Inspect feature
   │             "before all" hook in "Inspect feature":
   │
   │      ResponseError: security_exception
   │ 	Root causes:
   │ 		security_exception: action [indices:admin/create] is unauthorized for user [testing-internal] with effective roles [superuser] on restricted indices [.ml-config], this action is granted by the index privileges [create_index,manage,all]
   │       at SniffingTransport._request (node_modules/@elastic/transport/src/Transport.ts:602:17)
   │       at processTicksAndRejections (node:internal/process/task_queues:95:5)
   │       at /Users/ireneblanco/Desktop/Projects/kibana/node_modules/@elastic/transport/src/Transport.ts:708:22
   │       at SniffingTransport.request (node_modules/@elastic/transport/src/Transport.ts:705:14)
   │       at Indices.create (node_modules/@elastic/elasticsearch/src/api/api/indices.ts:290:12)
   │       at attemptToCreate (create_index_stream.ts:162:9)
   │       at handleIndex (create_index_stream.ts:228:5)
   │       at Transform.transform [as _transform] (create_index_stream.ts:238:13)
   │
   │
   │2)    Serverless Observability - Deployment-agnostic api integration tests
   │       APM
   │         inspect
   │           Inspect feature
   │             "after all" hook in "Inspect feature":
   │
   │      ResponseError: security_exception
   │ 	Root causes:
   │ 		security_exception: action [indices:admin/delete] is unauthorized for user [testing-internal] with effective roles [superuser] on restricted indices [.ml-config], this action is granted by the index privileges [delete_index,manage,all]
   │       at SniffingTransport._request (node_modules/@elastic/transport/src/Transport.ts:602:17)
   │       at processTicksAndRejections (node:internal/process/task_queues:95:5)
   │       at /Users/ireneblanco/Desktop/Projects/kibana/node_modules/@elastic/transport/src/Transport.ts:708:22
   │       at SniffingTransport.request (node_modules/@elastic/transport/src/Transport.ts:705:14)
   │       at Indices.delete (node_modules/@elastic/elasticsearch/src/api/api/indices.ts:394:12)
   │       at deleteIndex (delete_index.ts:45:5)
   │       at Transform.transform [as _transform] (delete_index_stream.ts:53:13)
   │
   │

cc: @crespocarlos

@iblancof
Copy link
Contributor

Reopening since we can run the tests with 8.0.0 instead of apm_8.0.0 (as suggested in #199817 (comment)), which includes the .ml-config index and isn’t compatible with serverless.

@iblancof iblancof reopened this Nov 13, 2024
@iblancof
Copy link
Contributor

The APM inspect test suite relies on Agent Configuration, which, based on my checks, isn’t available in Serverless. Therefore, we can’t migrate this to be deployment-agnostic.

I’ll exclude the test using Agent Configuration from the migration; the rest can be migrated using 8.0.0 instead of apm_8.0.0.

@iblancof iblancof mentioned this issue Nov 13, 2024
Merged
4 tasks
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 14, 2024
@iblancof
[APM] Migrate inspect API tests to be deployment-agnostic (elastic#19…
a896a98 
…9963)

### How to test
Closes elastic#198977
Part of elastic#193245

This PR contains the changes to migrate `inspect` test folder to
deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

(cherry picked from commit bb68f92)
wayneseymour pushed a commit to wayneseymour/kibana that referenced this issue Nov 18, 2024
@iblancof @wayneseymour
[APM] Migrate inspect API tests to be deployment-agnostic (elastic#19…
b4c63b6 
…9963)

### How to test
Closes elastic#198977
Part of elastic#193245

This PR contains the changes to migrate `inspect` test folder to
deployment-agnostic testing strategy.

### How to test


- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Nov 18, 2024
@iblancof @CAWilson94
[APM] Migrate inspect API tests to be deployment-agnostic (elastic#19…
d38b45a 
…9963)

### How to test
Closes elastic#198977
Part of elastic#193245

This PR contains the changes to migrate `inspect` test folder to
deployment-agnostic testing strategy.

### How to test


- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Nov 18, 2024
@iblancof @CAWilson94
[APM] Migrate inspect API tests to be deployment-agnostic (elastic#19…
6731dc9 
…9963)

### How to test
Closes elastic#198977
Part of elastic#193245

This PR contains the changes to migrate `inspect` test folder to
deployment-agnostic testing strategy.

### How to test


- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@iblancof iblancof

Labels
Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team technical debt Improvement of the software architecture and operational architecture
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[APM] Migrate inspect API tests to be deployment-agnostic iblancof/kibana
2 participants
@rmyz @iblancof