[APM][Otel] Add Otel client based on PoC data

[jennypavlova]

Closes #192115
Closes #192465

Summary

This PR adds synthrace client for Otel native data and a simple scenario. This is the first step of adding it and in the future it will include more metrics and use cases.

Note

To run ES the command needs "xpack.otel_data.registry.enabled=true" flag
yarn es snapshot --license trial --E "xpack.otel_data.registry.enabled=true"

Next steps

• We currently have only service_destination in the metrics indices we can include the other types in the future
• After we have all the UI changes we can add more scenarios (also using the opentelemetry demo data and not only the e2e PoC example)

Testing

• Run ES:

yarn es snapshot --license trial --E "xpack.otel_data.registry.enabled=true"

• Run Kibana:

yarn start

Warning

If the e2e PoC is used the first 2 steps should be skipped

• Run syntrace:

node scripts/synthtrace otel_simple_trace.ts --clean

• Check indices in DevTools for the generated data:

GET *metrics-generic.otel*/_search

GET *traces-generic.otel*/_search

GET *logs-generic.otel*/_search

• Check in the APM UI (all the tabs)

Warning

Currently the UI changes done in APM are not merged so some errors are expected)

Screen.Recording.2024-10-07.at.20.52.34.mov

[obltmachine]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[MiriamAparicio]

@gregkalapos can you review this PR and check that the fields and everything looks good, @jennypavlova and I used the Otel documents on the POC as a source of truth but just to double-check 🙏

[cauemarcondes]

LGTM

[MiriamAparicio]

LGTM 🌟
Thanks for the team work 💪

[gregkalapos]

Some findings below regarding fields.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 7382cc8
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-192293-7382cc8529a5

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/apm-synthtrace	64	67	+3
@kbn/apm-synthtrace-client	223	240	+17
total			+20

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
@kbn/apm-synthtrace	10	11	+1

Unknown metric groups

API count

id	before	after	diff
@kbn/apm-synthtrace	64	67	+3
@kbn/apm-synthtrace-client	223	240	+17
total			+20

History

• 💔 Build #241748 failed 18446fd
• 💚 Build #241711 succeeded 59fa9fd
• 💚 Build #240316 succeeded 6a59171
• 💚 Build #240242 succeeded d68e935
• 💚 Build #240240 succeeded 832abf1

cc @jennypavlova