Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for replaying to OTLP endpoints #45

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

endorama
Copy link
Member

This PR enhance apmsoak with the capability to replay otel data to OTLP backend endpoints.

Creating sample files is possible using the (experimental) otel File Exporter. At the moment this PR is using sample data from here.

There have been some challenges in adapting apmsoak for the OTLP use case:

  1. the code was expecting a single endpoint where to replay all data types (logs, metrics, traces). This is not the case for OTLP where each data type has it's own endpoint. This has been overcome through conventions in the value assigned to agent_name config in scenario file: agents starting with apm are considered Elastic APM agents whereas agents starting with otlp are considered differently. By matching agent name for otlp-logs, otlp-metrics and otlp-traces is also possible to select the appropriate data to replay.
  2. OTLP File Exporter suggest to using jsonlines (.jsonl) file format; we are using ndjson. The 2 formats should be compatible but I haven't fully investigated the details. OTLP sample files are sources from .jsonl files.
  3. otlp/grpc is not supported.

Closes #18

@endorama
Copy link
Member Author

At the moment I'm stuck debugging the handler logic around creating batches, which for the OTLP case produce a segfault. I suspect this is due to the different data format on which the same transform operations don't work as expected.

(note to self, why not implementing otlpsoak, only dedicated to otlp?)

@endorama
Copy link
Member Author

I extracted the relevant refactoring to accomodate this functionality in #46. This PR will be based on top of that branch, with additional commits to implement OTLP support.

@endorama
Copy link
Member Author

Rebase completed, this branch now only include changes relevant to the OTLP support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: add support for otlp endpoints
1 participant