Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM] Foundation for migrating APM tests to deployment agnostic approach #198775

Conversation

crespocarlos
Copy link
Contributor

@crespocarlos crespocarlos commented Nov 4, 2024

part of #193245
closes #198958

Summary

This PR lays the foundation to start migrating APM API integration tests to the deployment agnostic approach.

How to test

  • Serverless
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"

It's recommended to be run against MKI

  • Stateful
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"

@crespocarlos crespocarlos force-pushed the 193245-apm-deployment-agnostic-tests-part-1 branch 9 times, most recently from 2393507 to 5277183 Compare November 4, 2024 13:25
Migrate agent_explorer test

Update latest_agent_versions test
@crespocarlos crespocarlos force-pushed the 193245-apm-deployment-agnostic-tests-part-1 branch from 5277183 to eee7cd3 Compare November 4, 2024 13:26
@crespocarlos
Copy link
Contributor Author

/ci

@crespocarlos
Copy link
Contributor Author

/ci

@crespocarlos crespocarlos changed the title [APM] Deployment agnostic tests part 1 [APM] Foundation for migrating APM tests to deployment agnostic approach Nov 4, 2024
@crespocarlos
Copy link
Contributor Author

/ci

@crespocarlos
Copy link
Contributor Author

@elasticmachine merge upstream

@crespocarlos
Copy link
Contributor Author

/ci

@kpatticha
Copy link
Contributor

@crespocarlos can you please add in the description how to run these tests. 🙏

@crespocarlos
Copy link
Contributor Author

@crespocarlos can you please add in the description how to run these tests. 🙏

Will do when I open the PR for review :)

@crespocarlos crespocarlos added release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Nov 5, 2024
@crespocarlos crespocarlos marked this pull request as ready for review November 5, 2024 15:09
@crespocarlos crespocarlos requested a review from a team as a code owner November 5, 2024 15:09
Copy link
Member

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great progress, Carlos 🚀

Thanks a lot for reaching us and making changes. I left some questions

// Skipping here will skip the entire apm api test suite
// Instead skip (flaky) tests individually
// Failing: See https://github.com/elastic/kibana/issues/176948
describe('APM API tests', function () {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally up to you about the naming, just since it is located in x-pack/test/api_integration you can probably omit API word.

Copy link
Member

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, but have to double check about Infra tests update.

@crespocarlos
Copy link
Contributor Author

Sorry, but have to double check about Infra tests update.

#198775 (comment)

@crespocarlos
Copy link
Contributor Author

@elasticmachine merge upstream

Copy link
Member

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

On behalf of Appex-QA, thank you for being open to feedback and doing few rounds of changes

@crespocarlos crespocarlos enabled auto-merge (squash) November 6, 2024 14:58
@elasticmachine
Copy link
Contributor

elasticmachine commented Nov 6, 2024

💚 Build Succeeded

  • Buildkite Build
  • Commit: 47795a1
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-198775-47795a1db0a9

Metrics [docs]

✅ unchanged

History

@crespocarlos crespocarlos merged commit 926861b into elastic:main Nov 6, 2024
23 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11707967174

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.x Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 198775

Questions ?

Please refer to the Backport tool documentation

mgadewoll pushed a commit to mgadewoll/kibana that referenced this pull request Nov 7, 2024
…ach (elastic#198775)

part of elastic#193245
closes elastic#198958

## Summary

This PR lays the foundation to start migrating APM API integration tests
to the deployment agnostic approach.


### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"
```

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 7, 2024
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 198775 locally

1 similar comment
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 198775 locally

crespocarlos added a commit to crespocarlos/kibana that referenced this pull request Nov 11, 2024
…ach (elastic#198775)

part of elastic#193245
closes elastic#198958

## Summary

This PR lays the foundation to start migrating APM API integration tests
to the deployment agnostic approach.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"
```

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 926861b)
crespocarlos added a commit that referenced this pull request Nov 11, 2024
… approach (#198775) (#199637)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[APM] Foundation for migrating APM tests to deployment agnostic
approach (#198775)](#198775)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Carlos
Crespo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-06T16:38:17Z","message":"[APM]
Foundation for migrating APM tests to deployment agnostic approach
(#198775)\n\npart of
https://github.com/elastic/kibana/issues/193245\r\ncloses
https://github.com/elastic/kibana/issues/198958\r\n\r\n##
Summary\r\n\r\nThis PR lays the foundation to start migrating APM API
integration tests\r\nto the deployment agnostic approach.\r\n\r\n\r\n###
How to test\r\n\r\n- Serverless\r\n\r\n```\r\nnode
scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
--grep=\"APM API tests\"\r\n```\r\n\r\nIt's recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n-
Stateful\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
--grep=\"APM API tests\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Elastic Machine
<[email protected]>","sha":"926861bcbce9a775eda7a389acedf935612b5985","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport
missing","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services"],"number":198775,"url":"https://github.com/elastic/kibana/pull/198775","mergeCommit":{"message":"[APM]
Foundation for migrating APM tests to deployment agnostic approach
(#198775)\n\npart of
https://github.com/elastic/kibana/issues/193245\r\ncloses
https://github.com/elastic/kibana/issues/198958\r\n\r\n##
Summary\r\n\r\nThis PR lays the foundation to start migrating APM API
integration tests\r\nto the deployment agnostic approach.\r\n\r\n\r\n###
How to test\r\n\r\n- Serverless\r\n\r\n```\r\nnode
scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
--grep=\"APM API tests\"\r\n```\r\n\r\nIt's recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n-
Stateful\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
--grep=\"APM API tests\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Elastic Machine
<[email protected]>","sha":"926861bcbce9a775eda7a389acedf935612b5985"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/198775","number":198775,"mergeCommit":{"message":"[APM]
Foundation for migrating APM tests to deployment agnostic approach
(#198775)\n\npart of
https://github.com/elastic/kibana/issues/193245\r\ncloses
https://github.com/elastic/kibana/issues/198958\r\n\r\n##
Summary\r\n\r\nThis PR lays the foundation to start migrating APM API
integration tests\r\nto the deployment agnostic approach.\r\n\r\n\r\n###
How to test\r\n\r\n- Serverless\r\n\r\n```\r\nnode
scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
--grep=\"APM API tests\"\r\n```\r\n\r\nIt's recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n-
Stateful\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
--grep=\"APM API tests\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Elastic Machine
<[email protected]>","sha":"926861bcbce9a775eda7a389acedf935612b5985"}}]}]
BACKPORT-->
@kibanamachine kibanamachine added v8.17.0 and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[APM] Migrate /test/apm_api_integration/tests/agent_explorer to be deployment-agnostic API tests
6 participants