Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[APM] Deployment-agnostic API tests #193245

Closed
crespocarlos opened this issue Sep 18, 2024 · 4 comments
Closed

[APM] Deployment-agnostic API tests #193245

crespocarlos opened this issue Sep 18, 2024 · 4 comments
Assignees
@crespocarlos
Labels
Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team technical debt Improvement of the software architecture and operational architecture

Comments

@crespocarlos
Copy link
Contributor

crespocarlos commented Sep 18, 2024 •
edited by iblancof
Loading

Summary

We need to make APM API tests deployment agnostic to dedup test cases as much as possible.
https://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/README.md

Important

APM tests are license-based, with most running on basic and trial licenses. To make these tests deployment-agnostic, there can't be any license-specific tests. The deployment-agnostic framework sets the license to trial, so when migrating the tests, we need to consider the following:

  • Happy path tests configured to run on both basic and trial can be migrated.
  • Tests that use the cloud license will not be migrated.
  • Tests that validate feature availability based on the license will not be migrated.
  • Deployment agnostic tests can only work with 3 roles: viewer, editor and admin. Any tests that rely on custom roles won't be migrated
  • When migrating tests that depend on archives, note that:
    • On the mappings.json, settings.lifecycle must be removed as there is no ILM concept in serverless.
    • Archives that create .alert-* indices can't be used in serveless because .alerts indices are created as managed Data Streams and the es_archiver fails to create those indices. Therefore, tests using it can't be migrated.
    • Archives that create .ml* indices can't be used in serverless. Here is the list of APM archives with .ml indices:
      • apm_8.0.0
      • ml_8.0.0

Migrated tests must be validated against MKI - https://github.com/elastic/observability-dev/issues/3519#issuecomment-2379914274
In case a test is unskipped, make sure to run the flaky test runner https://ci-stats.kibana.dev/trigger_flaky_test_runner/1

API ESS Serverless Test files
GET /internal/apm/get_agents_per_service ✅ ❌ ESS: ./test/apm_api_integration/tests/agent_explorer/agent_explorer.spec.ts
GET /internal/apm/get_latest_agent_versions ✅ ❌ ESS: ./test/apm_api_integration/tests/agent_explorer/latest_agent_versions.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/alerts/anomaly_alert.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/alerts/error_count_threshold.spec.ts
GET /internal/apm/rule_types/error_count/chart_preview ✅ ❌ ESS: ./test/apm_api_integration/tests/alerts/preview_chart_error_count.spec.ts
GET /internal/apm/rule_types/transaction_error_rate/chart_preview ✅ ❌ ESS: ./test/apm_api_integration/tests/alerts/preview_chart_error_rate.spec.ts
GET /internal/apm/rule_types/transaction_duration/chart_preview ✅ ❌ ESS: ./test/apm_api_integration/tests/alerts/preview_chart_transaction_duration.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/alerts/transaction_duration.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/alerts/transaction_error_rate.spec.ts
GET /internal/apm/services/{serviceName}/anomaly_charts ✅ ❌ ESS: ./test/apm_api_integration/tests/anomalies/anomaly_charts.spec.ts
GET /internal/apm/services/{serviceName}/transactions/charts/coldstart_rate ✅ ❌ ESS: ./test/apm_api_integration/tests/cold_start/cold_start.spec.ts
GET /internal/apm/services/{serviceName}/transactions/charts/coldstart_rate_by_transaction_name ✅ ❌ ESS: ./test/apm_api_integration/tests/cold_start/cold_start_by_transaction_name/cold_start_by_transaction_name.spec.ts
POST /internal/apm/latency/overall_distribution/transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/correlations/failed_transactions.spec.ts
GET /internal/apm/correlations/field_candidates/transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/correlations/field_candidates.spec.ts
POST /internal/apm/correlations/field_value_pairs/transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/correlations/field_value_pairs.spec.ts
POST /internal/apm/latency/overall_distribution/transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/correlations/latency.spec.ts
POST /internal/apm/correlations/p_values/transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/correlations/p_values.spec.ts
POST /internal/apm/correlations/significant_correlations/transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/correlations/significant_correlations.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/custom_dashboards/custom_dashboards.spec.ts
POST /internal/apm/data_view/static ✅ ❌ ESS: ./test/apm_api_integration/tests/data_view/static.spec.ts
GET /internal/apm/dependencies/charts/{metric} ✅ ❌ ESS: ./test/apm_api_integration/tests/dependencies/dependency_metrics.spec.ts
GET /internal/apm/dependencies/metadata ✅ ❌ ESS: ./test/apm_api_integration/tests/dependencies/metadata.spec.ts
GET /internal/apm/services/{serviceName}/dependencies ✅ ❌ ESS: ./test/apm_api_integration/tests/dependencies/service_dependencies.spec.ts
GET /internal/apm/dependencies/top_dependencies ✅ ❌ ESS: ./test/apm_api_integration/tests/dependencies/top_dependencies.spec.ts
GET /internal/apm/dependencies/operations ✅ ❌ ESS: ./test/apm_api_integration/tests/dependencies/top_operations.spec.ts
GET /internal/apm/dependencies/operations/spans ✅ ❌ ESS: ./test/apm_api_integration/tests/dependencies/top_spans.spec.ts
GET /internal/apm/dependencies/upstream_services ✅ ❌ ESS: ./test/apm_api_integration/tests/dependencies/upstream_services.spec.ts
GET /internal/apm/diagnostics ✅ ❌ ESS: ./test/apm_api_integration/tests/diagnostics/apm_events.spec.ts
GET /internal/apm/diagnostics ✅ ❌ ESS: ./test/apm_api_integration/tests/diagnostics/data_streams.spec.ts
GET /internal/apm/diagnostics ✅ ❌ ESS: ./test/apm_api_integration/tests/diagnostics/index_templates.spec.ts
GET /internal/apm/diagnostics ✅ ❌ ESS: ./test/apm_api_integration/tests/diagnostics/indices.spec.ts
GET /internal/apm/diagnostics ✅ ❌ ESS: ./test/apm_api_integration/tests/diagnostics/privileges.spec.ts
GET /internal/apm/entities/services/{serviceName}/logs_error_rate_timeseries ✅ ❌ ESS: ./test/apm_api_integration/tests/entities/logs/service_logs_error_rate_timeseries.spec.ts
GET /internal/apm/entities/services/{serviceName}/logs_rate_timeseries ✅ ❌ ESS: ./test/apm_api_integration/tests/entities/logs/service_logs_rate_timeseries.spec.ts
GET /internal/apm/environments ✅ ❌ ESS: ./test/apm_api_integration/tests/environment/get_environment.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/error_rate/service_apis.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/error_rate/service_maps.spec.ts
GET /internal/apm/services/{serviceName}/errors/distribution ✅ ❌ ESS: ./test/apm_api_integration/tests/errors/distribution.spec.ts
GET /internal/apm/services/{serviceName}/errors/groups/main_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/errors/error_group_list.spec.ts
GET /internal/apm/services/{serviceName}/errors/{groupId}/samples ✅ ❌ ESS: ./test/apm_api_integration/tests/errors/group_id_samples.spec.ts
GET /internal/apm/services/{serviceName}/errors/{groupId}/top_erroneous_transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/errors/top_erroneous_transactions/top_erroneous_transactions.spec.ts
GET /internal/apm/services/{serviceName}/errors/groups/main_statistics_by_transaction_name ✅ ❌ ESS: ./test/apm_api_integration/tests/errors/top_errors_for_transaction/top_errors_main_stats.spec.ts
GET /internal/apm/event_metadata/{processorEvent}/{id} ✅ ❌ ESS: ./test/apm_api_integration/tests/event_metadata/event_metadata.spec.ts
PUT /api/apm/settings/agent-configuration ✅ ❌ ESS: ./test/apm_api_integration/tests/feature_controls.spec.ts
PUT /api/apm/settings/agent-configuration ✅ ❌ ESS: ./test/apm_api_integration/tests/fleet/apm_package_policy.spec.ts
GET /internal/apm/services ✅ ❌ ESS: ./test/apm_api_integration/tests/fleet/input_only_package.spec.ts
GET /internal/apm/fleet/migration_check ✅ ❌ ESS: ./test/apm_api_integration/tests/fleet/migration_check.spec.ts
GET /internal/apm/has_data ✅ ❌ ESS: ./test/apm_api_integration/tests/historical_data/has_data.spec.ts
GET /internal/apm/services/{serviceName}/infrastructure_attributes ✅ ❌ ESS: ./test/apm_api_integration/tests/infrastructure/infrastructure_attributes.spec.ts
GET /internal/apm/environments ✅ ❌ ESS: ./test/apm_api_integration/tests/inspect/inspect.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/latency/service_apis.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/latency/service_maps.spec.ts
GET /internal/apm/services/{serviceName}/metrics/charts ✅ ❌ ESS: ./test/apm_api_integration/tests/metrics/memory/memory_metrics.spec.ts
GET /internal/apm/services/{serviceName}/metrics/charts ✅ ❌ ESS: ./test/apm_api_integration/tests/metrics/metrics_charts.spec.ts
GET /internal/apm/services/{serviceName}/metrics/serverless/active_instances,` ✅ ❌ ESS: ./test/apm_api_integration/tests/metrics/serverless/serverless_active_instances.spec.ts
GET /internal/apm/services/{serviceName}/metrics/serverless/functions_overview ✅ ❌ ESS: ./test/apm_api_integration/tests/metrics/serverless/serverless_functions_overview.spec.ts
GET /internal/apm/services/{serviceName}/metrics/serverless/charts ✅ ❌ ESS: ./test/apm_api_integration/tests/metrics/serverless/serverless_metrics_charts.spec.ts
GET /internal/apm/services/{serviceName}/metrics/serverless/summary ✅ ❌ ESS: ./test/apm_api_integration/tests/metrics/serverless/serverless_summary.spec.ts
GET /internal/apm/mobile-services/{serviceName}/crashes/groups/main_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/crashes/crash_group_list.spec.ts
GET /internal/apm/mobile-services/{serviceName}/crashes/distribution ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/crashes/distribution.spec.ts
GET /internal/apm/services/{serviceName}/errors/{groupId}/samples ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/errors/group_id_samples.spec.ts
GET /internal/apm/mobile-services/{serviceName}/detailed_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_detailed_statistics_by_field.spec.ts
GET /internal/apm/services/{serviceName}/mobile/filters ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_filters.spec.ts
GET /internal/apm/mobile-services/{serviceName}/transactions/charts/http_requests ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_http_requests_timeseries.spec.ts
GET /internal/apm/mobile-services/{serviceName}/location/stats ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_location_stats.spec.ts
GET /internal/apm/mobile-services/{serviceName}/main_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_main_statistics_by_field.spec.ts
GET /internal/apm/mobile-services/{serviceName}/most_used_charts ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_most_used_chart.spec.ts
GET /internal/apm/mobile-services/{serviceName}/transactions/charts/sessions ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_sessions_timeseries.spec.ts
GET /internal/apm/mobile-services/{serviceName}/stats ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_stats.spec.ts
GET /internal/apm/mobile-services/{serviceName}/terms ✅ ❌ ESS: ./test/apm_api_integration/tests/mobile/mobile_terms_by_field.spec.ts
GET /internal/apm/observability_overview/has_data ✅ ❌ ESS: ./test/apm_api_integration/tests/observability_overview/has_data.spec.ts
GET /internal/apm/services ✅ ❌ ESS: ./test/apm_api_integration/tests/observability_overview/observability_overview.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/service_groups/save_service_group.spec.ts
- ✅ ❌ ESS: ./test/apm_api_integration/tests/service_groups/service_group_count/service_group_count.spec.ts
GET /internal/apm/services ✅ ❌ ESS: ./test/apm_api_integration/tests/service_groups/service_group_with_overflow/service_group_with_overflow.spec.ts
GET /internal/apm/service-map ✅ ✅ Different implementation in both environments.
ESS: ./test/apm_api_integration/tests/service_maps/service_maps.spec.ts
Serverless: x-pack/test_serverless/api_integration/test_suites/observability/apm_api_integration/service_maps/service_maps.ts
GET /internal/apm/service-map ✅ ❌ ESS: ./test/apm_api_integration/tests/service_maps/service_maps_kuery_filter.spec.ts
GET /internal/apm/services/{serviceName}/metrics/nodes ✅ ❌ ESS: ./test/apm_api_integration/tests/service_nodes/get_service_nodes.spec.ts
GET /internal/apm/services/{serviceName}/dependencies ✅ ❌ ESS: ./test/apm_api_integration/tests/service_overview/dependencies/index.spec.ts
GET /internal/apm/services/{serviceName}/service_overview_instances/details/{serviceNodeName} ✅ ❌ ESS: ./test/apm_api_integration/tests/service_overview/instance_details.spec.ts
GET /internal/apm/services/{serviceName}/service_overview_instances/detailed_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/service_overview/instances_detailed_statistics.spec.ts
GET /internal/apm/services/{serviceName}/service_overview_instances/main_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/service_overview/instances_main_statistics.spec.ts
GET /internal/apm/services/{serviceName}/agent ✅ ❌ ESS: ./test/apm_api_integration/tests/services/agent.spec.ts
POST /api/apm/services/{serviceName}/annotation ✅ ❌ ESS: ./test/apm_api_integration/tests/services/annotations.spec.ts
POST /internal/apm/services/detailed_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/services/archive_services_detailed_statistics.spec.ts
GET /api/apm/services/{serviceName}/annotation/search ✅ ❌ ESS: ./test/apm_api_integration/tests/services/derived_annotations.spec.ts
POST /internal/apm/services/{serviceName}/errors/groups/detailed_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/services/error_groups/error_groups_detailed_statistics.spec.ts
GET /internal/apm/services/{serviceName}/errors/groups/main_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/services/error_groups/error_groups_main_statistics.spec.ts
GET /internal/apm/services/{serviceName}/node/{serviceNodeName}/metadata ✅ ❌ ESS: ./test/apm_api_integration/tests/services/get_service_node_metadata.spec.ts
GET /internal/apm/services/{serviceName}/alerts_count ✅ ❌ ESS: ./test/apm_api_integration/tests/services/service_alerts.spec.ts
GET /internal/apm/services/{serviceName}/metadata/details ✅ ❌ ESS: ./test/apm_api_integration/tests/services/service_details/service_details.spec.ts
GET /internal/apm/services/{serviceName}/service_overview_instances/details/{serviceNodeName} ✅ ❌ ESS: ./test/apm_api_integration/tests/services/service_details/service_infra_metrics.spec.ts
GET /internal/apm/services/{serviceName}/metadata/icons ✅ ❌ ESS: ./test/apm_api_integration/tests/services/service_icons/service_icons.spec.ts
POST /internal/apm/services/detailed_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/services/services_detailed_statistics.spec.ts
GET /internal/apm/services/{serviceName}/throughput ✅ ❌ ESS: ./test/apm_api_integration/tests/services/throughput.spec.ts
GET /internal/apm/services ✅ ❌ ESS: ./test/apm_api_integration/tests/services/top_services.spec.ts
GET /internal/apm/services/{serviceName}/transaction_types ✅ ❌ ESS: ./test/apm_api_integration/tests/services/transaction_types.spec.ts
GET /api/apm/settings/agent-configuration/environments ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/agent_configuration/agent_configuration.spec.ts
POST /api/apm/agent_keys ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/agent_keys/agent_keys.spec.ts
GET /internal/apm/settings/anomaly-detection/jobs ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/anomaly_detection/basic.spec.ts
GET /internal/apm/settings/anomaly-detection/jobs ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/anomaly_detection/no_access_user.spec.ts
GET /internal/apm/settings/anomaly-detection/jobs ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/anomaly_detection/read_user.spec.ts
GET /internal/apm/settings/anomaly-detection/jobs ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/anomaly_detection/update_to_v3.spec.ts
GET /internal/apm/settings/anomaly-detection/jobs ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/anomaly_detection/write_user.spec.ts
GET /internal/apm/settings/apm-indices ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/apm_indices/apm_indices.spec.ts
GET /internal/apm/settings/custom_links/transaction ✅ ❌ ESS: ./test/apm_api_integration/tests/settings/custom_link/custom_link.spec.ts
GET /internal/apm/traces/{traceId} ✅ ❌ ESS: ./test/apm_api_integration/tests/span_links/span_links.spec.ts
GET /internal/apm/storage_explorer/get_services ✅ ❌ ESS: ./test/apm_api_integration/tests/storage_explorer/get_services.spec.ts
GET /internal/apm/services/{serviceName}/storage_details ✅ ❌ ESS: ./test/apm_api_integration/tests/storage_explorer/storage_details.spec.ts
GET /internal/apm/storage_explorer ✅ ❌ ESS: ./test/apm_api_integration/tests/storage_explorer/storage_explorer.spec.ts
GET /internal/apm/storage_explorer/privileges ✅ ❌ ESS: ./test/apm_api_integration/tests/storage_explorer/storage_explorer_privileges.spec.ts
GET /internal/apm/storage_explorer_summary_stats ✅ ❌ ESS: ./test/apm_api_integration/tests/storage_explorer/storage_explorer_summary_stats.spec.ts
GET /internal/apm/storage_chart ✅ ❌ ESS: ./test/apm_api_integration/tests/storage_explorer/storage_explorer_timeseries_chart.spec.ts
GET /internal/apm/suggestions ✅ ❌ ESS: ./test/apm_api_integration/tests/suggestions/suggestions.spec.ts
GET /internal/apm/dependencies/top_dependencies ✅ ❌ ESS: ./test/apm_api_integration/tests/throughput/dependencies_apis.spec.ts
GET /internal/apm/services ✅ ❌ ESS: ./test/apm_api_integration/tests/throughput/service_apis.spec.ts
GET /internal/apm/service-map/service/{serviceName} ✅ ❌ ESS: ./test/apm_api_integration/tests/throughput/service_maps.spec.ts
GET /internal/apm/time_range_metadata ✅ ❌ ESS: ./test/apm_api_integration/tests/time_range_metadata/many_apm_server_versions.spec.ts
GET /internal/apm/time_range_metadata ✅ ❌ ESS: ./test/apm_api_integration/tests/time_range_metadata/time_range_metadata.spec.ts
POST /internal/apm/traces/aggregated_critical_path ✅ ✅ Different implementation in both environments.
ESS: ./test/apm_api_integration/tests/traces/critical_path.spec.ts
Serverless: x-pack/test_serverless/api_integration/test_suites/observability/apm_api_integration/traces/critical_path.ts
GET /internal/apm/traces/find ✅ ❌ ESS: ./test/apm_api_integration/tests/traces/find_traces.spec.ts
GET /internal/apm/traces/{traceId} ✅ ❌ ESS: ./test/apm_api_integration/tests/traces/large_trace/large_trace.spec.ts
GET /internal/apm/traces/{traceId}/spans/{spanId} ✅ ❌ ESS: ./test/apm_api_integration/tests/traces/span_details.spec.ts
GET /internal/apm/traces ✅ ❌ ESS: ./test/apm_api_integration/tests/traces/top_traces.spec.ts
GET /internal/apm/traces/{traceId} ✅ ❌ ESS: ./test/apm_api_integration/tests/traces/trace_by_id.spec.ts
GET /internal/apm/traces/{traceId}/transactions/{transactionId} ✅ ❌ ESS: ./test/apm_api_integration/tests/traces/transaction_details.spec.ts
GET /internal/apm/services/{serviceName}/transaction/charts/breakdown ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/breakdown.spec.ts
GET /internal/apm/services/{serviceName}/transactions/charts/error_rate ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/error_rate.spec.ts
GET /internal/apm/services/{serviceName}/transactions/charts/latency ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/latency.spec.ts
POST /internal/apm/latency/overall_distribution/transactions ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/latency_overall_distribution.spec.ts
GET /internal/apm/services/{serviceName}/transactions/traces/samples ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/trace_samples.spec.ts
GET /internal/apm/services/{serviceName}/transactions/groups/main_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/transactions_groups_alerts.spec.ts
GET /internal/apm/services/{serviceName}/transactions/groups/detailed_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/transactions_groups_detailed_statistics.spec.ts
GET /internal/apm/services/{serviceName}/transactions/groups/main_statistics ✅ ❌ ESS: ./test/apm_api_integration/tests/transactions/transactions_groups_main_statistics.spec.ts
- ❌ ✅ Serverless: x-pack/test_serverless/api_integration/test_suites/observability/apm_api_integration/feature_flags.ts
@crespocarlos crespocarlos added technical debt Improvement of the software architecture and operational architecture needs-refinement A reason and acceptance criteria need to be defined for this issue Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Sep 18, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

@crespocarlos crespocarlos removed the needs-refinement A reason and acceptance criteria need to be defined for this issue label Sep 18, 2024
@crespocarlos crespocarlos changed the title [Infra][APM] Deployment-agnostic API tests [APM] Deployment-agnostic API tests Oct 28, 2024
@crespocarlos crespocarlos self-assigned this Oct 30, 2024
@crespocarlos crespocarlos mentioned this issue Nov 5, 2024
Merged
@kpatticha kpatticha mentioned this issue Nov 5, 2024
Closed
crespocarlos added a commit that referenced this issue Nov 6, 2024
@crespocarlos @kibanamachine @elasticmachine
[APM] Foundation for migrating APM tests to deployment agnostic appro…
926861b 
…ach (#198775)

part of #193245
closes #198958

## Summary

This PR lays the foundation to start migrating APM API integration tests
to the deployment agnostic approach.


### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"
```

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
This was referenced Nov 7, 2024
Closed
Merged
Merged
mgadewoll pushed a commit to mgadewoll/kibana that referenced this issue Nov 7, 2024
@crespocarlos @kibanamachine
@elasticmachine @mgadewoll
[APM] Foundation for migrating APM tests to deployment agnostic appro…
4d69748 
…ach (elastic#198775)

part of elastic#193245
closes elastic#198958

## Summary

This PR lays the foundation to start migrating APM API integration tests
to the deployment agnostic approach.


### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"
```

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
@rmyz rmyz mentioned this issue Nov 7, 2024
Merged
4 tasks
@miloszmarcinkowski miloszmarcinkowski mentioned this issue Nov 7, 2024
Merged
@crespocarlos crespocarlos mentioned this issue Nov 7, 2024
Merged
miloszmarcinkowski added a commit that referenced this issue Nov 8, 2024
@miloszmarcinkowski
Migrate /custom_dashboards to be deployment agnostic (#199295)
a947525 
closes #198964 
closes #198966
part of #193245

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- MKI

tested against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)
This was referenced Nov 9, 2024
Merged
Merged
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Nov 11, 2024
@miloszmarcinkowski @CAWilson94
Migrate /custom_dashboards to be deployment agnostic (elastic#199295)
0c34369 
closes elastic#198964 
closes elastic#198966
part of elastic#193245

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- MKI

tested against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)
crespocarlos added a commit to crespocarlos/kibana that referenced this issue Nov 11, 2024
@crespocarlos
[APM] Foundation for migrating APM tests to deployment agnostic appro…
af568d6 
…ach (elastic#198775)

part of elastic#193245
closes elastic#198958

## Summary

This PR lays the foundation to start migrating APM API integration tests
to the deployment agnostic approach.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"
```

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 926861b)
@miloszmarcinkowski miloszmarcinkowski mentioned this issue Nov 11, 2024
Merged
4 tasks
miloszmarcinkowski added a commit to miloszmarcinkowski/kibana that referenced this issue Nov 11, 2024
@miloszmarcinkowski
Migrate /custom_dashboards to be deployment agnostic (elastic#199295)
5da1d2a 
closes elastic#198964
closes elastic#198966
part of elastic#193245

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- MKI

tested against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

(cherry picked from commit a947525)
rmyz added a commit that referenced this issue Nov 11, 2024
@rmyz
[APM] Migrate /cold_start to deployment agnostic test (#199270)
d27f241 
## Summary

Closes #198961
Part of #193245

This PR contains the changes to migrate `cold_start` test folder to
Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless
@iblancof iblancof mentioned this issue Nov 11, 2024
Closed
@MiriamAparicio MiriamAparicio mentioned this issue Nov 11, 2024
Merged
4 tasks
rmyz added a commit that referenced this issue Nov 11, 2024
@rmyz
[APM] Migrate /entities to deployment agnostic test (#199579)
c611e52 
## Summary

Closes #198968
Part of #193245

This PR contains the changes to migrate `entities` test folder to
Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless
rmyz added a commit to rmyz/kibana that referenced this issue Nov 11, 2024
@rmyz
[APM] Migrate /cold_start to deployment agnostic test (elastic#199270)
ce2944c 
## Summary

Closes elastic#198961
Part of elastic#193245

This PR contains the changes to migrate `cold_start` test folder to
Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

(cherry picked from commit d27f241)
rmyz added a commit to rmyz/kibana that referenced this issue Nov 11, 2024
@rmyz
[APM] Migrate /entities to deployment agnostic test (elastic#199579)
3fa5573 
## Summary

Closes elastic#198968
Part of elastic#193245

This PR contains the changes to migrate `entities` test folder to
Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

(cherry picked from commit c611e52)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 22, 2024
@crespocarlos
[APM] Migrate traces tests to deployment agnostic (elastic#200561)
316f51b 
## Summary

Closes [elastic#198995](elastic#198995)
Closes elastic#198996
Part of elastic#193245

This PR contains the changes to migrate `traces` test folder to
Deployment-agnostic testing strategy.
It also adds support for deployment-agnostic snapshots validation

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- [ ] ~(OPTIONAL, only if a test has been unskipped) Run flaky test
suite~
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

---------

Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 3917d02)
@Bluefinger Bluefinger mentioned this issue Nov 22, 2024
Merged
Bluefinger added a commit that referenced this issue Nov 25, 2024
@Bluefinger @jennypavlova @elasticmachine
[ObsUX][APM] Migrate Service Overview archiver test cases to synthtra…
4891c7d 
…ce (#201407)

Part of #193245
Related #200226
Closes #200743 

## Summary

This PR completes the migration of remaining Service Overview tests to
the Deployment Agnostic test framework. In this PR, one test was
deduplicated (the Dependencies test), and the Instances Detailed
Statistics cases dealing with archiver data was migrated to make use of
synthtrace instead. Snapshots included were redone to match the data
generated by synthtrace, but no other cases were changed to ensure the
new migrated tests were passing the same assumptions as before.

## How to Test

### Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Stateful

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
```

---------

Co-authored-by: jennypavlova <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 25, 2024
@Bluefinger
[ObsUX][APM] Migrate Service Overview archiver test cases to synthtra…
1ee4dc0 
…ce (elastic#201407)

Part of elastic#193245
Related elastic#200226
Closes elastic#200743

## Summary

This PR completes the migration of remaining Service Overview tests to
the Deployment Agnostic test framework. In this PR, one test was
deduplicated (the Dependencies test), and the Instances Detailed
Statistics cases dealing with archiver data was migrated to make use of
synthtrace instead. Snapshots included were redone to match the data
generated by synthtrace, but no other cases were changed to ensure the
new migrated tests were passing the same assumptions as before.

## How to Test

### Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Stateful

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
```

---------

Co-authored-by: jennypavlova <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 4891c7d)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 25, 2024
@Bluefinger
[ObsUX][APM] Migrate Service Overview archiver test cases to synthtra…
24f34f9 
…ce (elastic#201407)

Part of elastic#193245
Related elastic#200226
Closes elastic#200743

## Summary

This PR completes the migration of remaining Service Overview tests to
the Deployment Agnostic test framework. In this PR, one test was
deduplicated (the Dependencies test), and the Instances Detailed
Statistics cases dealing with archiver data was migrated to make use of
synthtrace instead. Snapshots included were redone to match the data
generated by synthtrace, but no other cases were changed to ensure the
new migrated tests were passing the same assumptions as before.

## How to Test

### Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Stateful

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
```

---------

Co-authored-by: jennypavlova <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 4891c7d)
dmlemeshko added a commit that referenced this issue Nov 26, 2024
@dmlemeshko
[8.x] [ftr][deployment-agnostic] move APM tests in its own config file (
e1691eb 
#200562) (#200924)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ftr][deployment-agnostic] move APM tests in its own config file
(#200562)](#200562)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-20T09:09:44Z","message":"[ftr][deployment-agnostic]
move APM tests in its own config file (#200562)\n\n##
Summary\r\n\r\nPart of #199182\r\n\r\nWith great progress of #193245 the
number of tests in Observability\r\ndeployment-agnostic config files is
growing and so does config run time\r\nby reaching **30**
minutes.\r\n\r\nAs we try to keep pipeline runtime reasonable, this PR
adds a new\r\nconfigs `oblt.apm.stateful.config.ts`
and\r\n`oblt.apm.serverless.config.ts` that load only APM-related tests
from\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\r\n\r\nIt
should help to speed up execution, we will double check
config\r\nruntime when APM test migration is completed.\r\n\r\nFor
reviewers: no extra work is expected from Oblt teams if new tests\r\nare
still imported
in\r\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`","sha":"a85127605c7bf7e0e349bc7bb687b69f72b6fe18","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","FTR","backport:version","v8.17.0"],"number":200562,"url":"https://github.com/elastic/kibana/pull/200562","mergeCommit":{"message":"[ftr][deployment-agnostic]
move APM tests in its own config file (#200562)\n\n##
Summary\r\n\r\nPart of #199182\r\n\r\nWith great progress of #193245 the
number of tests in Observability\r\ndeployment-agnostic config files is
growing and so does config run time\r\nby reaching **30**
minutes.\r\n\r\nAs we try to keep pipeline runtime reasonable, this PR
adds a new\r\nconfigs `oblt.apm.stateful.config.ts`
and\r\n`oblt.apm.serverless.config.ts` that load only APM-related tests
from\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\r\n\r\nIt
should help to speed up execution, we will double check
config\r\nruntime when APM test migration is completed.\r\n\r\nFor
reviewers: no extra work is expected from Oblt teams if new tests\r\nare
still imported
in\r\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`","sha":"a85127605c7bf7e0e349bc7bb687b69f72b6fe18"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/200562","number":200562,"mergeCommit":{"message":"[ftr][deployment-agnostic]
move APM tests in its own config file (#200562)\n\n##
Summary\r\n\r\nPart of #199182\r\n\r\nWith great progress of #193245 the
number of tests in Observability\r\ndeployment-agnostic config files is
growing and so does config run time\r\nby reaching **30**
minutes.\r\n\r\nAs we try to keep pipeline runtime reasonable, this PR
adds a new\r\nconfigs `oblt.apm.stateful.config.ts`
and\r\n`oblt.apm.serverless.config.ts` that load only APM-related tests
from\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\r\n\r\nIt
should help to speed up execution, we will double check
config\r\nruntime when APM test migration is completed.\r\n\r\nFor
reviewers: no extra work is expected from Oblt teams if new tests\r\nare
still imported
in\r\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`","sha":"a85127605c7bf7e0e349bc7bb687b69f72b6fe18"}},{"branch":"8.x","label":"v8.17.0","labelRegex":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this issue Nov 26, 2024
@dmlemeshko @paulinashakirova
[ftr][deployment-agnostic] move APM tests in its own config file (ela…
7872c15 
…stic#200562)

## Summary

Part of elastic#199182

With great progress of elastic#193245 the number of tests in Observability
deployment-agnostic config files is growing and so does config run time
by reaching **30** minutes.

As we try to keep pipeline runtime reasonable, this PR adds a new
configs `oblt.apm.stateful.config.ts` and
`oblt.apm.serverless.config.ts` that load only APM-related tests from
https://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts

It should help to speed up execution, we will double check config
runtime when APM test migration is completed.

For reviewers: no extra work is expected from Oblt teams if new tests
are still imported in
`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this issue Nov 26, 2024
@Bluefinger @kibanamachine
@elasticmachine @paulinashakirova
[ObsUX][APM] Migration of Service Overview tests to deployment agnost…
601ad4b 
…ic approach (elastic#200226)

## Summary

Part of elastic#193245
Closes elastic#198986

This PR moves all compatible/supported test cases for Service Overview.
Unsupported cases are kept in the old test section to run on stateful
for now.

## How to Test

### Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Stateful

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"
```

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this issue Nov 26, 2024
@rmyz @paulinashakirova
[APM] Migrate /transactions to deployment agnostic test (elastic#20…
00ff9d8 
…0694)

## Summary

Closes elastic#198997
Part of elastic#193245

This PR contains the changes to migrate `transactions` test folder to
Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
```

It's recommended to be run against
[MKI](https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
```

## Checks

- [x] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless 

<!--ONMERGE {"backportTargets":["8.x"]} ONMERGE-->
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this issue Nov 26, 2024
@iblancof @crespocarlos @paulinashakirova
[APM] Migrate settings API tests to be deployment-agnostic (elastic#2…
cbed4f0 
…00762)

Closes elastic#198989
Part of elastic#193245

This PR contains the changes to migrate `settings` test folder to
deployment-agnostic testing strategy.


**Not Migrated**
- `agent_configuration`: Not available in Serverless.
- `anomaly_detection/no_access`: Involves the `noAccess` user role; we
are only migrating tests for `viewer`, `editor`, or `admin` roles.
- `anomaly_detection/update_to_v3`: Involves the deletion of ML jobs; we
will wait until an "ml" service is available to properly migrate these
tests.
- `anomaly_detection/write_user`: Involves the deletion of ML jobs; we
will wait until an "ml" service is available to properly migrate these
tests.

**Partially Migrated**
- `anomaly_detection/read_user`: Involves the
`apmAllPrivilegesWithoutWriteSettingsUser` role; only tests for the
`read` role have been migrated.
- `anomaly_detection/write_user`: Involves the
`apmReadPrivilegesWithWriteSettingsUser` role; only tests for the
`write` role have been migrated.
- `apm_indices`: Tests based on license have not been migrated.
custom_link: Involves the `apmReadPrivilegesWithWriteSettingsUser` role;
only tests for the trial `write` role have been migrated.
- `agent_keys`: Involves the `manageOwnAgentKeysUser` and
`createAndAllAgentKeysUser` roles; only tests for the `write` role have
been migrated.

### How to test


- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts 
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts 
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts  
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless 

<!--ONMERGE {"backportTargets":["8.x"]} ONMERGE-->

---------

Co-authored-by: Carlos Crespo <[email protected]>
Co-authored-by: Carlos Crespo <[email protected]>
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this issue Nov 26, 2024
@crespocarlos @elasticmachine @paulinashakirova
[APM] Migrate traces tests to deployment agnostic (elastic#200561)
ca22be0 
## Summary

Closes [elastic#198995](elastic#198995)
Closes elastic#198996
Part of elastic#193245

This PR contains the changes to migrate `traces` test folder to
Deployment-agnostic testing strategy.
It also adds support for deployment-agnostic snapshots validation

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- [ ] ~(OPTIONAL, only if a test has been unskipped) Run flaky test
suite~
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

---------

Co-authored-by: Elastic Machine <[email protected]>
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this issue Nov 26, 2024
@Bluefinger @jennypavlova
@elasticmachine @paulinashakirova
[ObsUX][APM] Migrate Service Overview archiver test cases to synthtra…
aaf3335 
…ce (elastic#201407)

Part of elastic#193245
Related elastic#200226
Closes elastic#200743 

## Summary

This PR completes the migration of remaining Service Overview tests to
the Deployment Agnostic test framework. In this PR, one test was
deduplicated (the Dependencies test), and the Instances Detailed
Statistics cases dealing with archiver data was migrated to make use of
synthtrace instead. Snapshots included were redone to match the data
generated by synthtrace, but no other cases were changed to ensure the
new migrated tests were passing the same assumptions as before.

## How to Test

### Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Stateful

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
```

---------

Co-authored-by: jennypavlova <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
kibanamachine added a commit that referenced this issue Dec 9, 2024
@kibanamachine @Bluefinger
[8.x] [ObsUX][APM] Migrate Service Overview archiver test cases to sy…
f1b5a91 
…nthtrace (#201407) (#201537)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ObsUX][APM] Migrate Service Overview archiver test cases to
synthtrace (#201407)](#201407)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Gonçalo Rica Pais da
Silva","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-25T10:13:16Z","message":"[ObsUX][APM]
Migrate Service Overview archiver test cases to synthtrace
(#201407)\n\nPart of #193245\r\nRelated #200226\r\nCloses #200743
\r\n\r\n## Summary\r\n\r\nThis PR completes the migration of remaining
Service Overview tests to\r\nthe Deployment Agnostic test framework. In
this PR, one test was\r\ndeduplicated (the Dependencies test), and the
Instances Detailed\r\nStatistics cases dealing with archiver data was
migrated to make use of\r\nsynthtrace instead. Snapshots included were
redone to match the data\r\ngenerated by synthtrace, but no other cases
were changed to ensure the\r\nnew migrated tests were passing the same
assumptions as before.\r\n\r\n## How to Test\r\n\r\n###
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\n```\r\n\r\nIt's
recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n###
Stateful\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
jennypavlova <[email protected]>\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"4891c7d7f99097de649667ad0c1f48515120edbd","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services","v8.17.0"],"title":"[ObsUX][APM]
Migrate Service Overview archiver test cases to
synthtrace","number":201407,"url":"https://github.com/elastic/kibana/pull/201407","mergeCommit":{"message":"[ObsUX][APM]
Migrate Service Overview archiver test cases to synthtrace
(#201407)\n\nPart of #193245\r\nRelated #200226\r\nCloses #200743
\r\n\r\n## Summary\r\n\r\nThis PR completes the migration of remaining
Service Overview tests to\r\nthe Deployment Agnostic test framework. In
this PR, one test was\r\ndeduplicated (the Dependencies test), and the
Instances Detailed\r\nStatistics cases dealing with archiver data was
migrated to make use of\r\nsynthtrace instead. Snapshots included were
redone to match the data\r\ngenerated by synthtrace, but no other cases
were changed to ensure the\r\nnew migrated tests were passing the same
assumptions as before.\r\n\r\n## How to Test\r\n\r\n###
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\n```\r\n\r\nIt's
recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n###
Stateful\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
jennypavlova <[email protected]>\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"4891c7d7f99097de649667ad0c1f48515120edbd"}},"sourceBranch":"main","suggestedTargetBranches":["8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201407","number":201407,"mergeCommit":{"message":"[ObsUX][APM]
Migrate Service Overview archiver test cases to synthtrace
(#201407)\n\nPart of #193245\r\nRelated #200226\r\nCloses #200743
\r\n\r\n## Summary\r\n\r\nThis PR completes the migration of remaining
Service Overview tests to\r\nthe Deployment Agnostic test framework. In
this PR, one test was\r\ndeduplicated (the Dependencies test), and the
Instances Detailed\r\nStatistics cases dealing with archiver data was
migrated to make use of\r\nsynthtrace instead. Snapshots included were
redone to match the data\r\ngenerated by synthtrace, but no other cases
were changed to ensure the\r\nnew migrated tests were passing the same
assumptions as before.\r\n\r\n## How to Test\r\n\r\n###
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\n```\r\n\r\nIt's
recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n###
Stateful\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
jennypavlova <[email protected]>\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"4891c7d7f99097de649667ad0c1f48515120edbd"}},{"branch":"8.17","label":"v8.17.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Gonçalo Rica Pais da Silva <[email protected]>
Bluefinger pushed a commit to Bluefinger/kibana that referenced this issue Dec 9, 2024
@crespocarlos
[APM] Migrate traces tests to deployment agnostic (elastic#200561)
95aa509 
## Summary

Closes [elastic#198995](elastic#198995)
Closes elastic#198996
Part of elastic#193245

This PR contains the changes to migrate `traces` test folder to
Deployment-agnostic testing strategy.
It also adds support for deployment-agnostic snapshots validation

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- [ ] ~(OPTIONAL, only if a test has been unskipped) Run flaky test
suite~
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

---------

Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 3917d02)
dmlemeshko added a commit to dmlemeshko/kibana that referenced this issue Dec 11, 2024
@dmlemeshko
[8.x] [ftr][deployment-agnostic] move APM tests in its own config file (
84d172a 
elastic#200562) (elastic#200924)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ftr][deployment-agnostic] move APM tests in its own config file
(elastic#200562)](elastic#200562)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-20T09:09:44Z","message":"[ftr][deployment-agnostic]
move APM tests in its own config file (elastic#200562)\n\n##
Summary\r\n\r\nPart of elastic#199182\r\n\r\nWith great progress of elastic#193245 the
number of tests in Observability\r\ndeployment-agnostic config files is
growing and so does config run time\r\nby reaching **30**
minutes.\r\n\r\nAs we try to keep pipeline runtime reasonable, this PR
adds a new\r\nconfigs `oblt.apm.stateful.config.ts`
and\r\n`oblt.apm.serverless.config.ts` that load only APM-related tests
from\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\r\n\r\nIt
should help to speed up execution, we will double check
config\r\nruntime when APM test migration is completed.\r\n\r\nFor
reviewers: no extra work is expected from Oblt teams if new tests\r\nare
still imported
in\r\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`","sha":"a85127605c7bf7e0e349bc7bb687b69f72b6fe18","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","FTR","backport:version","v8.17.0"],"number":200562,"url":"https://github.com/elastic/kibana/pull/200562","mergeCommit":{"message":"[ftr][deployment-agnostic]
move APM tests in its own config file (elastic#200562)\n\n##
Summary\r\n\r\nPart of elastic#199182\r\n\r\nWith great progress of elastic#193245 the
number of tests in Observability\r\ndeployment-agnostic config files is
growing and so does config run time\r\nby reaching **30**
minutes.\r\n\r\nAs we try to keep pipeline runtime reasonable, this PR
adds a new\r\nconfigs `oblt.apm.stateful.config.ts`
and\r\n`oblt.apm.serverless.config.ts` that load only APM-related tests
from\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\r\n\r\nIt
should help to speed up execution, we will double check
config\r\nruntime when APM test migration is completed.\r\n\r\nFor
reviewers: no extra work is expected from Oblt teams if new tests\r\nare
still imported
in\r\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`","sha":"a85127605c7bf7e0e349bc7bb687b69f72b6fe18"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/200562","number":200562,"mergeCommit":{"message":"[ftr][deployment-agnostic]
move APM tests in its own config file (elastic#200562)\n\n##
Summary\r\n\r\nPart of elastic#199182\r\n\r\nWith great progress of elastic#193245 the
number of tests in Observability\r\ndeployment-agnostic config files is
growing and so does config run time\r\nby reaching **30**
minutes.\r\n\r\nAs we try to keep pipeline runtime reasonable, this PR
adds a new\r\nconfigs `oblt.apm.stateful.config.ts`
and\r\n`oblt.apm.serverless.config.ts` that load only APM-related tests
from\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\r\n\r\nIt
should help to speed up execution, we will double check
config\r\nruntime when APM test migration is completed.\r\n\r\nFor
reviewers: no extra work is expected from Oblt teams if new tests\r\nare
still imported
in\r\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`","sha":"a85127605c7bf7e0e349bc7bb687b69f72b6fe18"}},{"branch":"8.x","label":"v8.17.0","labelRegex":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

(cherry picked from commit e1691eb)
dmlemeshko added a commit that referenced this issue Dec 11, 2024
@dmlemeshko
[8.17] [8.x] [ftr][deployment-agnostic] move APM tests in its own con…
86cbc85 
…fig file (#200562) (#200924) (#203728)

# Backport

This will backport the following commits from `8.x` to `8.17`:
- [[8.x] [ftr][deployment-agnostic] move APM tests in its own config
file (#200562) (#200924)](#200924)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-26T10:27:33Z","message":"[8.x]
[ftr][deployment-agnostic] move APM tests in its own config file
(#200562) (#200924)\n\n# Backport\n\nThis will backport the following
commits from `main` to `8.x`:\n- [[ftr][deployment-agnostic] move APM
tests in its own config
file\n(#200562)](https://github.com/elastic/kibana/pull/200562)\n\n<!---
Backport version: 8.9.8 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Dzmitry\nLemechko\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2024-11-20T09:09:44Z\",\"message\":\"[ftr][deployment-agnostic]\nmove
APM tests in its own config file
(#200562)\\n\\n##\nSummary\\r\\n\\r\\nPart of #199182\\r\\n\\r\\nWith
great progress of #193245 the\nnumber of tests in
Observability\\r\\ndeployment-agnostic config files is\ngrowing and so
does config run time\\r\\nby reaching **30**\nminutes.\\r\\n\\r\\nAs we
try to keep pipeline runtime reasonable, this PR\nadds a
new\\r\\nconfigs
`oblt.apm.stateful.config.ts`\nand\\r\\n`oblt.apm.serverless.config.ts`
that load only APM-related
tests\nfrom\\r\\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\\r\\n\\r\\nIt\nshould
help to speed up execution, we will double check\nconfig\\r\\nruntime
when APM test migration is completed.\\r\\n\\r\\nFor\nreviewers: no
extra work is expected from Oblt teams if new tests\\r\\nare\nstill
imported\nin\\r\\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`\",\"sha\":\"a85127605c7bf7e0e349bc7bb687b69f72b6fe18\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.17.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:skip\",\"v9.0.0\",\"FTR\",\"backport:version\",\"v8.17.0\"],\"number\":200562,\"url\":\"https://github.com/elastic/kibana/pull/200562\",\"mergeCommit\":{\"message\":\"[ftr][deployment-agnostic]\nmove
APM tests in its own config file
(#200562)\\n\\n##\nSummary\\r\\n\\r\\nPart of #199182\\r\\n\\r\\nWith
great progress of #193245 the\nnumber of tests in
Observability\\r\\ndeployment-agnostic config files is\ngrowing and so
does config run time\\r\\nby reaching **30**\nminutes.\\r\\n\\r\\nAs we
try to keep pipeline runtime reasonable, this PR\nadds a
new\\r\\nconfigs
`oblt.apm.stateful.config.ts`\nand\\r\\n`oblt.apm.serverless.config.ts`
that load only APM-related
tests\nfrom\\r\\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\\r\\n\\r\\nIt\nshould
help to speed up execution, we will double check\nconfig\\r\\nruntime
when APM test migration is completed.\\r\\n\\r\\nFor\nreviewers: no
extra work is expected from Oblt teams if new tests\\r\\nare\nstill
imported\nin\\r\\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`\",\"sha\":\"a85127605c7bf7e0e349bc7bb687b69f72b6fe18\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.x\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"labelRegex\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/200562\",\"number\":200562,\"mergeCommit\":{\"message\":\"[ftr][deployment-agnostic]\nmove
APM tests in its own config file
(#200562)\\n\\n##\nSummary\\r\\n\\r\\nPart of #199182\\r\\n\\r\\nWith
great progress of #193245 the\nnumber of tests in
Observability\\r\\ndeployment-agnostic config files is\ngrowing and so
does config run time\\r\\nby reaching **30**\nminutes.\\r\\n\\r\\nAs we
try to keep pipeline runtime reasonable, this PR\nadds a
new\\r\\nconfigs
`oblt.apm.stateful.config.ts`\nand\\r\\n`oblt.apm.serverless.config.ts`
that load only APM-related
tests\nfrom\\r\\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\\r\\n\\r\\nIt\nshould
help to speed up execution, we will double check\nconfig\\r\\nruntime
when APM test migration is completed.\\r\\n\\r\\nFor\nreviewers: no
extra work is expected from Oblt teams if new tests\\r\\nare\nstill
imported\nin\\r\\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`\",\"sha\":\"a85127605c7bf7e0e349bc7bb687b69f72b6fe18\"}},{\"branch\":\"8.x\",\"label\":\"v8.17.0\",\"labelRegex\":\"^v8.17.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->","sha":"e1691eb91d710e3ccbb1422176505bf306c8eb44","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["backport"],"number":200924,"url":"https://github.com/elastic/kibana/pull/200924","mergeCommit":{"message":"[8.x]
[ftr][deployment-agnostic] move APM tests in its own config file
(#200562) (#200924)\n\n# Backport\n\nThis will backport the following
commits from `main` to `8.x`:\n- [[ftr][deployment-agnostic] move APM
tests in its own config
file\n(#200562)](https://github.com/elastic/kibana/pull/200562)\n\n<!---
Backport version: 8.9.8 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Dzmitry\nLemechko\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2024-11-20T09:09:44Z\",\"message\":\"[ftr][deployment-agnostic]\nmove
APM tests in its own config file
(#200562)\\n\\n##\nSummary\\r\\n\\r\\nPart of #199182\\r\\n\\r\\nWith
great progress of #193245 the\nnumber of tests in
Observability\\r\\ndeployment-agnostic config files is\ngrowing and so
does config run time\\r\\nby reaching **30**\nminutes.\\r\\n\\r\\nAs we
try to keep pipeline runtime reasonable, this PR\nadds a
new\\r\\nconfigs
`oblt.apm.stateful.config.ts`\nand\\r\\n`oblt.apm.serverless.config.ts`
that load only APM-related
tests\nfrom\\r\\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\\r\\n\\r\\nIt\nshould
help to speed up execution, we will double check\nconfig\\r\\nruntime
when APM test migration is completed.\\r\\n\\r\\nFor\nreviewers: no
extra work is expected from Oblt teams if new tests\\r\\nare\nstill
imported\nin\\r\\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`\",\"sha\":\"a85127605c7bf7e0e349bc7bb687b69f72b6fe18\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.17.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:skip\",\"v9.0.0\",\"FTR\",\"backport:version\",\"v8.17.0\"],\"number\":200562,\"url\":\"https://github.com/elastic/kibana/pull/200562\",\"mergeCommit\":{\"message\":\"[ftr][deployment-agnostic]\nmove
APM tests in its own config file
(#200562)\\n\\n##\nSummary\\r\\n\\r\\nPart of #199182\\r\\n\\r\\nWith
great progress of #193245 the\nnumber of tests in
Observability\\r\\ndeployment-agnostic config files is\ngrowing and so
does config run time\\r\\nby reaching **30**\nminutes.\\r\\n\\r\\nAs we
try to keep pipeline runtime reasonable, this PR\nadds a
new\\r\\nconfigs
`oblt.apm.stateful.config.ts`\nand\\r\\n`oblt.apm.serverless.config.ts`
that load only APM-related
tests\nfrom\\r\\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\\r\\n\\r\\nIt\nshould
help to speed up execution, we will double check\nconfig\\r\\nruntime
when APM test migration is completed.\\r\\n\\r\\nFor\nreviewers: no
extra work is expected from Oblt teams if new tests\\r\\nare\nstill
imported\nin\\r\\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`\",\"sha\":\"a85127605c7bf7e0e349bc7bb687b69f72b6fe18\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.x\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"labelRegex\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/200562\",\"number\":200562,\"mergeCommit\":{\"message\":\"[ftr][deployment-agnostic]\nmove
APM tests in its own config file
(#200562)\\n\\n##\nSummary\\r\\n\\r\\nPart of #199182\\r\\n\\r\\nWith
great progress of #193245 the\nnumber of tests in
Observability\\r\\ndeployment-agnostic config files is\ngrowing and so
does config run time\\r\\nby reaching **30**\nminutes.\\r\\n\\r\\nAs we
try to keep pipeline runtime reasonable, this PR\nadds a
new\\r\\nconfigs
`oblt.apm.stateful.config.ts`\nand\\r\\n`oblt.apm.serverless.config.ts`
that load only APM-related
tests\nfrom\\r\\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts\\r\\n\\r\\nIt\nshould
help to speed up execution, we will double check\nconfig\\r\\nruntime
when APM test migration is completed.\\r\\n\\r\\nFor\nreviewers: no
extra work is expected from Oblt teams if new tests\\r\\nare\nstill
imported\nin\\r\\n`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`\",\"sha\":\"a85127605c7bf7e0e349bc7bb687b69f72b6fe18\"}},{\"branch\":\"8.x\",\"label\":\"v8.17.0\",\"labelRegex\":\"^v8.17.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->","sha":"e1691eb91d710e3ccbb1422176505bf306c8eb44"}},"sourceBranch":"8.x","suggestedTargetBranches":[],"targetPullRequestStates":[]}]
BACKPORT-->
Bluefinger added a commit that referenced this issue Dec 11, 2024
@Bluefinger @crespocarlos
[8.17] [APM] Migrate traces tests to deployment agnostic (#200561) (#…
4688d25 
…203475)

# Backport

This will backport the following commits from `main` to `8.17`:
- [[APM] Migrate traces tests to deployment agnostic
(#200561)](#200561)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Carlos
Crespo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-22T13:47:06Z","message":"[APM]
Migrate traces tests to deployment agnostic (#200561)\n\n##
Summary\r\n\r\nCloses
[#198995](https://github.com/elastic/kibana/issues/198995)\r\nCloses
https://github.com/elastic/kibana/issues/198996\r\nPart of
https://github.com/elastic/kibana/issues/193245\r\n\r\nThis PR contains
the changes to migrate `traces` test folder to\r\nDeployment-agnostic
testing strategy.\r\nIt also adds support for deployment-agnostic
snapshots validation\r\n\r\n### How to test\r\n\r\n-
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
--grep=\"APM\"\r\n```\r\n\r\nIt's recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n-
Stateful\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
--grep=\"APM\"\r\n```\r\n\r\n- [ ] ~(OPTIONAL, only if a test has been
unskipped) Run flaky test\r\nsuite~\r\n- [x] local run for
serverless\r\n- [x] local run for stateful\r\n- [x] MKI run for
serverless\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"3917d02c51e9c4a83013fe83c87b77fb12bd1fbb","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services","v8.18.0"],"number":200561,"url":"https://github.com/elastic/kibana/pull/200561","mergeCommit":{"message":"[APM]
Migrate traces tests to deployment agnostic (#200561)\n\n##
Summary\r\n\r\nCloses
[#198995](https://github.com/elastic/kibana/issues/198995)\r\nCloses
https://github.com/elastic/kibana/issues/198996\r\nPart of
https://github.com/elastic/kibana/issues/193245\r\n\r\nThis PR contains
the changes to migrate `traces` test folder to\r\nDeployment-agnostic
testing strategy.\r\nIt also adds support for deployment-agnostic
snapshots validation\r\n\r\n### How to test\r\n\r\n-
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
--grep=\"APM\"\r\n```\r\n\r\nIt's recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n-
Stateful\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
--grep=\"APM\"\r\n```\r\n\r\n- [ ] ~(OPTIONAL, only if a test has been
unskipped) Run flaky test\r\nsuite~\r\n- [x] local run for
serverless\r\n- [x] local run for stateful\r\n- [x] MKI run for
serverless\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"3917d02c51e9c4a83013fe83c87b77fb12bd1fbb"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/200561","number":200561,"mergeCommit":{"message":"[APM]
Migrate traces tests to deployment agnostic (#200561)\n\n##
Summary\r\n\r\nCloses
[#198995](https://github.com/elastic/kibana/issues/198995)\r\nCloses
https://github.com/elastic/kibana/issues/198996\r\nPart of
https://github.com/elastic/kibana/issues/193245\r\n\r\nThis PR contains
the changes to migrate `traces` test folder to\r\nDeployment-agnostic
testing strategy.\r\nIt also adds support for deployment-agnostic
snapshots validation\r\n\r\n### How to test\r\n\r\n-
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
--grep=\"APM\"\r\n```\r\n\r\nIt's recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n-
Stateful\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
--grep=\"APM\"\r\n```\r\n\r\n- [ ] ~(OPTIONAL, only if a test has been
unskipped) Run flaky test\r\nsuite~\r\n- [x] local run for
serverless\r\n- [x] local run for stateful\r\n- [x] MKI run for
serverless\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"3917d02c51e9c4a83013fe83c87b77fb12bd1fbb"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/201386","number":201386,"state":"MERGED","mergeCommit":{"sha":"e127a2902a0bb93731aa02d5825fbdeee60bdfb9","message":"[8.x]
[APM] Migrate traces tests to deployment agnostic (#200561)
(#201386)\n\n# Backport\n\nThis will backport the following commits from
`main` to `8.x`:\n- [[APM] Migrate traces tests to deployment
agnostic\n(#200561)](https://github.com/elastic/kibana/pull/200561)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Carlos\nCrespo\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2024-11-22T13:47:06Z\",\"message\":\"[APM]\nMigrate
traces tests to deployment agnostic
(#200561)\\n\\n##\nSummary\\r\\n\\r\\nCloses\n[#198995](https://github.com/elastic/kibana/issues/198995)\\r\\nCloses\nhttps://github.com/elastic/kibana/issues/198996\\r\\nPart
of\nhttps://github.com//issues/193245\\r\\n\\r\\nThis PR
contains\nthe changes to migrate `traces` test folder
to\\r\\nDeployment-agnostic\ntesting strategy.\\r\\nIt also adds support
for deployment-agnostic\nsnapshots validation\\r\\n\\r\\n### How to
test\\r\\n\\r\\n-\nServerless\\r\\n\\r\\n```\\r\\nnode
scripts/functional_tests_server
--config\nx-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\\r\\nnode\nscripts/functional_test_runner
--config\nx-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\n--grep=\\\"APM\\\"\\r\\n```\\r\\n\\r\\nIt's
recommended to be
run\nagainst\\r\\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\\r\\n\\r\\n-\nStateful\\r\\n```\\r\\nnode
scripts/functional_tests_server
--config\nx-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\\r\\nnode\nscripts/functional_test_runner
--config\nx-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\n--grep=\\\"APM\\\"\\r\\n```\\r\\n\\r\\n-
[ ] ~(OPTIONAL, only if a test has been\nunskipped) Run flaky
test\\r\\nsuite~\\r\\n- [x] local run for\nserverless\\r\\n- [x] local
run for stateful\\r\\n- [x] MKI run
for\nserverless\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by: Elastic
Machine\n<[email protected]>\",\"sha\":\"3917d02c51e9c4a83013fe83c87b77fb12bd1fbb\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:skip\",\"v9.0.0\",\"backport:prev-minor\",\"ci:project-deploy-observability\",\"Team:obs-ux-infra_services\"],\"title\":\"[APM]\nMigrate
traces tests to
deployment\nagnostic\",\"number\":200561,\"url\":\"https://github.com/elastic/kibana/pull/200561\",\"mergeCommit\":{\"message\":\"[APM]\nMigrate
traces tests to deployment agnostic
(#200561)\\n\\n##\nSummary\\r\\n\\r\\nCloses\n[#198995](https://github.com/elastic/kibana/issues/198995)\\r\\nCloses\nhttps://github.com/elastic/kibana/issues/198996\\r\\nPart
of\nhttps://github.com//issues/193245\\r\\n\\r\\nThis PR
contains\nthe changes to migrate `traces` test folder
to\\r\\nDeployment-agnostic\ntesting strategy.\\r\\nIt also adds support
for deployment-agnostic\nsnapshots validation\\r\\n\\r\\n### How to
test\\r\\n\\r\\n-\nServerless\\r\\n\\r\\n```\\r\\nnode
scripts/functional_tests_server
--config\nx-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\\r\\nnode\nscripts/functional_test_runner
--config\nx-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\n--grep=\\\"APM\\\"\\r\\n```\\r\\n\\r\\nIt's
recommended to be
run\nagainst\\r\\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\\r\\n\\r\\n-\nStateful\\r\\n```\\r\\nnode
scripts/functional_tests_server
--config\nx-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\\r\\nnode\nscripts/functional_test_runner
--config\nx-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\n--grep=\\\"APM\\\"\\r\\n```\\r\\n\\r\\n-
[ ] ~(OPTIONAL, only if a test has been\nunskipped) Run flaky
test\\r\\nsuite~\\r\\n- [x] local run for\nserverless\\r\\n- [x] local
run for stateful\\r\\n- [x] MKI run
for\nserverless\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by: Elastic
Machine\n<[email protected]>\",\"sha\":\"3917d02c51e9c4a83013fe83c87b77fb12bd1fbb\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/200561\",\"number\":200561,\"mergeCommit\":{\"message\":\"[APM]\nMigrate
traces tests to deployment agnostic
(#200561)\\n\\n##\nSummary\\r\\n\\r\\nCloses\n[#198995](https://github.com/elastic/kibana/issues/198995)\\r\\nCloses\nhttps://github.com/elastic/kibana/issues/198996\\r\\nPart
of\nhttps://github.com//issues/193245\\r\\n\\r\\nThis PR
contains\nthe changes to migrate `traces` test folder
to\\r\\nDeployment-agnostic\ntesting strategy.\\r\\nIt also adds support
for deployment-agnostic\nsnapshots validation\\r\\n\\r\\n### How to
test\\r\\n\\r\\n-\nServerless\\r\\n\\r\\n```\\r\\nnode
scripts/functional_tests_server
--config\nx-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\\r\\nnode\nscripts/functional_test_runner
--config\nx-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts\n--grep=\\\"APM\\\"\\r\\n```\\r\\n\\r\\nIt's
recommended to be
run\nagainst\\r\\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\\r\\n\\r\\n-\nStateful\\r\\n```\\r\\nnode
scripts/functional_tests_server
--config\nx-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\\r\\nnode\nscripts/functional_test_runner
--config\nx-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts\n--grep=\\\"APM\\\"\\r\\n```\\r\\n\\r\\n-
[ ] ~(OPTIONAL, only if a test has been\nunskipped) Run flaky
test\\r\\nsuite~\\r\\n- [x] local run for\nserverless\\r\\n- [x] local
run for stateful\\r\\n- [x] MKI run
for\nserverless\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by: Elastic
Machine\n<[email protected]>\",\"sha\":\"3917d02c51e9c4a83013fe83c87b77fb12bd1fbb\"}}]}]\nBACKPORT-->\n\nCo-authored-by:
Carlos Crespo <[email protected]>\nCo-authored-by:
Elastic Machine <[email protected]>"}}]}]
BACKPORT-->

Co-authored-by: Carlos Crespo <[email protected]>
kibanamachine added a commit that referenced this issue Dec 11, 2024
@kibanamachine @Bluefinger
[8.17] [ObsUX][APM] Migrate Service Overview archiver test cases to s…
fdad711 
…ynthtrace (#201407) (#201536)

# Backport

This will backport the following commits from `main` to `8.17`:
- [[ObsUX][APM] Migrate Service Overview archiver test cases to
synthtrace (#201407)](#201407)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Gonçalo Rica Pais da
Silva","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-25T10:13:16Z","message":"[ObsUX][APM]
Migrate Service Overview archiver test cases to synthtrace
(#201407)\n\nPart of #193245\r\nRelated #200226\r\nCloses #200743
\r\n\r\n## Summary\r\n\r\nThis PR completes the migration of remaining
Service Overview tests to\r\nthe Deployment Agnostic test framework. In
this PR, one test was\r\ndeduplicated (the Dependencies test), and the
Instances Detailed\r\nStatistics cases dealing with archiver data was
migrated to make use of\r\nsynthtrace instead. Snapshots included were
redone to match the data\r\ngenerated by synthtrace, but no other cases
were changed to ensure the\r\nnew migrated tests were passing the same
assumptions as before.\r\n\r\n## How to Test\r\n\r\n###
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\n```\r\n\r\nIt's
recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n###
Stateful\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
jennypavlova <[email protected]>\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"4891c7d7f99097de649667ad0c1f48515120edbd","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services","v8.17.0"],"title":"[ObsUX][APM]
Migrate Service Overview archiver test cases to
synthtrace","number":201407,"url":"https://github.com/elastic/kibana/pull/201407","mergeCommit":{"message":"[ObsUX][APM]
Migrate Service Overview archiver test cases to synthtrace
(#201407)\n\nPart of #193245\r\nRelated #200226\r\nCloses #200743
\r\n\r\n## Summary\r\n\r\nThis PR completes the migration of remaining
Service Overview tests to\r\nthe Deployment Agnostic test framework. In
this PR, one test was\r\ndeduplicated (the Dependencies test), and the
Instances Detailed\r\nStatistics cases dealing with archiver data was
migrated to make use of\r\nsynthtrace instead. Snapshots included were
redone to match the data\r\ngenerated by synthtrace, but no other cases
were changed to ensure the\r\nnew migrated tests were passing the same
assumptions as before.\r\n\r\n## How to Test\r\n\r\n###
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\n```\r\n\r\nIt's
recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n###
Stateful\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
jennypavlova <[email protected]>\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"4891c7d7f99097de649667ad0c1f48515120edbd"}},"sourceBranch":"main","suggestedTargetBranches":["8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201407","number":201407,"mergeCommit":{"message":"[ObsUX][APM]
Migrate Service Overview archiver test cases to synthtrace
(#201407)\n\nPart of #193245\r\nRelated #200226\r\nCloses #200743
\r\n\r\n## Summary\r\n\r\nThis PR completes the migration of remaining
Service Overview tests to\r\nthe Deployment Agnostic test framework. In
this PR, one test was\r\ndeduplicated (the Dependencies test), and the
Instances Detailed\r\nStatistics cases dealing with archiver data was
migrated to make use of\r\nsynthtrace instead. Snapshots included were
redone to match the data\r\ngenerated by synthtrace, but no other cases
were changed to ensure the\r\nnew migrated tests were passing the same
assumptions as before.\r\n\r\n## How to Test\r\n\r\n###
Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\n```\r\n\r\nIt's
recommended to be run
against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n###
Stateful\r\n\r\n```\r\nnode scripts/functional_tests_server --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\nnode
scripts/functional_test_runner --config
x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by:
jennypavlova <[email protected]>\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"4891c7d7f99097de649667ad0c1f48515120edbd"}},{"branch":"8.17","label":"v8.17.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Gonçalo Rica Pais da Silva <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@crespocarlos @kibanamachine @CAWilson94
[APM] Revert deletion of service agent test (elastic#200196)
e973c76 
Closes elastic#198988
Part of elastic#193245

## Summary

Follow up from elastic#199812. The
`agent.spec.ts` test was deleted by mistake.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- [ ] ~(OPTIONAL, only if a test has been unskipped) Run flaky test
suite~
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

---------

Co-authored-by: kibanamachine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@rmyz @kibanamachine @CAWilson94
[APM] Migrate /service_maps to deployment agnostic test (elastic#19…
8ae15bb 
…9984)

## Summary

Closes elastic#198983
Part of elastic#193245

This PR contains the changes to migrate `service_maps` test folder to
Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

---------

Co-authored-by: kibanamachine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@miloszmarcinkowski @kibanamachine @CAWilson94
Migrate /test/apm_api_integration/tests/span_links to be deployment-a…
0f83839 
…gnostic API tests (elastic#200140)

closes elastic#198990
part of elastic#193245

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

-
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Checklist
- [x] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] serverless
- [x] stateful
- [x] MKI

---------

Co-authored-by: kibanamachine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@crespocarlos @elasticmachine
@kibanamachine @CAWilson94
[APM] Migrate time_range_metadata test to deployment agnostic (elasti…
2f748e9 
…c#200146)

## Summary

Closes [elastic#198994](elastic#198994)
Part of elastic#193245

This PR contains the changes to migrate `time_range_metadata` test
folder to Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- [ ] ~(OPTIONAL, only if a test has been unskipped) Run flaky test
suite~
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

---------

Co-authored-by: Elastic Machine <[email protected]>
Co-authored-by: kibanamachine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@miloszmarcinkowski @rmyz @CAWilson94
Migrate /test/apm_api_integration/tests/suggestions to be deploymen…
273b4ae 
…t agnostic api tests (elastic#200556)

closes elastic#198992
closes elastic#198993
part of elastic#193245

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

-
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Checklist
- [x] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] serverless
- [x] stateful
- [x] MKI

---------

Co-authored-by: Sergi Romeu <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@dmlemeshko @CAWilson94
[ftr][deployment-agnostic] move APM tests in its own config file (ela…
4780592 
…stic#200562)

## Summary

Part of elastic#199182

With great progress of elastic#193245 the number of tests in Observability
deployment-agnostic config files is growing and so does config run time
by reaching **30** minutes.

As we try to keep pipeline runtime reasonable, this PR adds a new
configs `oblt.apm.stateful.config.ts` and
`oblt.apm.serverless.config.ts` that load only APM-related tests from
https://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts

It should help to speed up execution, we will double check config
runtime when APM test migration is completed.

For reviewers: no extra work is expected from Oblt teams if new tests
are still imported in
`x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/index.ts`
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@Bluefinger @kibanamachine
@elasticmachine @CAWilson94
[ObsUX][APM] Migration of Service Overview tests to deployment agnost…
2a61f1e 
…ic approach (elastic#200226)

## Summary

Part of elastic#193245
Closes elastic#198986

This PR moves all compatible/supported test cases for Service Overview.
Unsupported cases are kept in the old test section to run on stateful
for now.

## How to Test

### Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM API tests"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Stateful

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM API tests"
```

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@rmyz @CAWilson94
[APM] Migrate /transactions to deployment agnostic test (elastic#20…
ad93a7f 
…0694)

## Summary

Closes elastic#198997
Part of elastic#193245

This PR contains the changes to migrate `transactions` test folder to
Deployment-agnostic testing strategy.

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
```

It's recommended to be run against
[MKI](https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
```

## Checks

- [x] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless 

<!--ONMERGE {"backportTargets":["8.x"]} ONMERGE-->
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@iblancof @crespocarlos @CAWilson94
[APM] Migrate settings API tests to be deployment-agnostic (elastic#2…
1c059b0 
…00762)

Closes elastic#198989
Part of elastic#193245

This PR contains the changes to migrate `settings` test folder to
deployment-agnostic testing strategy.


**Not Migrated**
- `agent_configuration`: Not available in Serverless.
- `anomaly_detection/no_access`: Involves the `noAccess` user role; we
are only migrating tests for `viewer`, `editor`, or `admin` roles.
- `anomaly_detection/update_to_v3`: Involves the deletion of ML jobs; we
will wait until an "ml" service is available to properly migrate these
tests.
- `anomaly_detection/write_user`: Involves the deletion of ML jobs; we
will wait until an "ml" service is available to properly migrate these
tests.

**Partially Migrated**
- `anomaly_detection/read_user`: Involves the
`apmAllPrivilegesWithoutWriteSettingsUser` role; only tests for the
`read` role have been migrated.
- `anomaly_detection/write_user`: Involves the
`apmReadPrivilegesWithWriteSettingsUser` role; only tests for the
`write` role have been migrated.
- `apm_indices`: Tests based on license have not been migrated.
custom_link: Involves the `apmReadPrivilegesWithWriteSettingsUser` role;
only tests for the trial `write` role have been migrated.
- `agent_keys`: Involves the `manageOwnAgentKeysUser` and
`createAndAllAgentKeysUser` roles; only tests for the `write` role have
been migrated.

### How to test


- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts 
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts 
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts  
```

## Checks

- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless 

<!--ONMERGE {"backportTargets":["8.x"]} ONMERGE-->

---------

Co-authored-by: Carlos Crespo <[email protected]>
Co-authored-by: Carlos Crespo <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@crespocarlos @elasticmachine @CAWilson94
[APM] Migrate traces tests to deployment agnostic (elastic#200561)
42d0ce0 
## Summary

Closes [elastic#198995](elastic#198995)
Closes elastic#198996
Part of elastic#193245

This PR contains the changes to migrate `traces` test folder to
Deployment-agnostic testing strategy.
It also adds support for deployment-agnostic snapshots validation

### How to test

- Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

- Stateful
```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"
```

- [ ] ~(OPTIONAL, only if a test has been unskipped) Run flaky test
suite~
- [x] local run for serverless
- [x] local run for stateful
- [x] MKI run for serverless

---------

Co-authored-by: Elastic Machine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@Bluefinger @jennypavlova
@elasticmachine @CAWilson94
[ObsUX][APM] Migrate Service Overview archiver test cases to synthtra…
2cbb1a8 
…ce (elastic#201407)

Part of elastic#193245
Related elastic#200226
Closes elastic#200743 

## Summary

This PR completes the migration of remaining Service Overview tests to
the Deployment Agnostic test framework. In this PR, one test was
deduplicated (the Dependencies test), and the Instances Detailed
Statistics cases dealing with archiver data was migrated to make use of
synthtrace instead. Snapshots included were redone to match the data
generated by synthtrace, but no other cases were changed to ensure the
new migrated tests were passing the same assumptions as before.

## How to Test

### Serverless

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt/apm.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts
```

It's recommended to be run against
[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)

### Stateful

```
node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts
```

---------

Co-authored-by: jennypavlova <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@crespocarlos crespocarlos

Labels
Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team technical debt Improvement of the software architecture and operational architecture
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@crespocarlos @Bluefinger @elasticmachine @iblancof