Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Obs AI Assistant] Remove commented out langtrace initialization and … #193381

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

anuraaga
Copy link

…make classes agnostic to used conventions.

Summary

Note, this PR changes absolutely no behavior, it is purely to try to make the code clearer for future improvements.

As discussed offline, it should be fine to remove the commented out initialization of langtrace SDK (we already do not have a dependency in package.json) and we can flesh out our story for initializing OpenTelemetry instrumentation in this component going forward. Note that the features of the langtrace SDK are

  • Initialize OpenTelemetry SDK with langtrace exporter. Note that while the default langtrace exporter is a custom protocol, langtrace also supports OTLP
  • Initialize langtrace instrumentation. We currently have our own instrumentation so shouldn't need this. Going forward we will want to think about whether we want to continue with the instrumentation implemented in kibana, replace it with langtrace instrumentation, or some other such as possible opentelemetry upstream ones

Especially because of the second point, we can consider enabling the OTel SDK directly without the langtrace SDK. But to repeat, this PR doesn't make any changes to behavior such as enabling the SDK, we can do it later.

In addition to removing the commented out code, it also renames some entry points with langtrace naming to be a more generic genai, e.g. LangTracer -> GenAITracer. This can make clearer that the actual instrumentation code is not using langtrace code but is using the OpenTelemetry API. Conventions, including attributes like llm.model are based on older langtrace versions, but langtrace itself has also moved towards converging on OpenTelemetry semconv, so going forward we will probably want to do the same.

/cc @codefromthecrypt @dgieselaar

Checklist

Just removing commented out code and pure renaming.

Risk Matrix

Just removing commented out code and pure renaming.

For maintainers

@anuraaga anuraaga requested a review from a team as a code owner September 19, 2024 04:02
@botelastic botelastic bot added ci:project-deploy-observability Create an Observability project Team:Obs AI Assistant Observability AI Assistant labels Sep 19, 2024
@codefromthecrypt
Copy link

non-binding comment: I had no idea the dep was already not in package.json. I like the idea of clarifying the code in use, and doing this prior to changing the semantics. I suspect even if we did want to use langtrace with the data produced here, we would need to change this data as they already have upstream to the otel semantics in 1.27.0.

@anuraaga
Copy link
Author

Ah just to clarify to reduce any confusion, there is a dep on langtrace, but it is only used for the attributes schema, not the SDK itself. And as this PR doesn't mean to change any attributes, I have left that for now, but as you mention we will likely want to move towards the OTel conventions.

https://github.com/elastic/kibana/blob/main/package.json#L996

@codefromthecrypt
Copy link

non-binding suggestion is to localize the constants and remove the dep? bc seems it has several deps of its own, so maybe can prove we aren't affected by those?

@anuraaga
Copy link
Author

Good idea, we actually were mostly just using it for schema validation (using the typescript type) with a couple of values for events. I inlined those and removed the dep, especially since we're pretty confident we'll need to update these to the latest spec that's both OTel and langtrace.

@codefromthecrypt
Copy link

great to see more lines gone than present in the last commit!

@kibana-ci
Copy link
Collaborator

kibana-ci commented Sep 19, 2024

💛 Build succeeded, but was flaky

  • Buildkite Build
  • Commit: cbc3c60
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-193381-cbc3c6006128

Failed CI Steps

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@karthikscale3
Copy link

Langtrace core maintainer here. Happy to prioritize anything needed to make things easier for you all. Feel free to open an issue here

@codefromthecrypt
Copy link

@karthikscale3 thanks for all the support as always. Luckily you are one of the few re-using the otel specs for core genai signals, so hopefully even custom code will work out against the langtrace service. we'll keep reaching out if any bumps happen in pursuit of that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci:project-deploy-observability Create an Observability project Team:Obs AI Assistant Observability AI Assistant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants