Add LogRecordProcessor to record event log records as span events #1551

jack-berg · 2024-11-18T19:42:23Z

Resolves open-telemetry/opentelemetry-java#6880.

Supersedes open-telemetry/opentelemetry-java#6650.

The event {@link LogRecordData} is converted to attributes on the span event as follows:

event.name attribute is mapped to span event name
LogRecordData#getTimestampEpochNanos() is mapped to span event timestamp
LogRecordData#getAttributes() are mapped to span event attributes, excluding event.name
LogRecordData#getObservedTimestampEpochNanos() is mapped to span event attribute with key log.record.observed_timestamp
LogRecordData#getSeverity() is mapped to span event attribute with key log.record.severity_number
LogRecordData#getBodyValue() is mapped to span event attribute with key log.record.body, as an escaped JSON string following the standard protobuf JSON encoding. NOTE: this results in an encoding that feels more verbose than necessary, but allows us to avoid having to define a new encoding.
LogRecordData#getTotalAttributeCount() - LogRecordData#getAttributes().size() is mapped to span event attribute log.record.dropped_attributes_count

jack-berg · 2024-11-18T19:46:23Z

@breedx-splk, @LikeTheSalad - I'm happy to become a component owner for io.opentelemetry.contrib:opentelemetry-processors.

…y-java-contrib into event-to-span-event-bridge

breedx-splk

Thanks. This is helpful and I didn't expect it to show up so soon! I had a few small questions/comments but this seems solid.

breedx-splk · 2024-11-18T21:29:03Z

processors/README.md

+    - event_to_span_event_bridge: {}
+```
+
+// TODO(jack-berg): remove "{}" after merging / rle https://github.com/open-telemetry/opentelemetry-java/pull/6891/files


I think this comment (assuming it stays here temporarily) ought to be inside the yaml block.

breedx-splk · 2024-11-18T21:30:09Z

processors/build.gradle.kts

+
+  // For EventToSpanEventBridge
+  implementation("io.opentelemetry:opentelemetry-exporter-otlp-common")
+  implementation("com.fasterxml.jackson.core:jackson-core")


Where is jackson used? I don't see it. Is it a transitive runtime-only dependency maybe?

Yeah its required to call Marshaler#writeJsonTo(OutputStream)

opentelemetry-exporter-logging-otlp has the same dependency here: https://github.com/open-telemetry/opentelemetry-java/blob/main/exporters/logging-otlp/build.gradle.kts#L19

Ok, that's what I was guessing. Can we change this to runtimeOnly() then?

breedx-splk · 2024-11-18T21:35:06Z

processors/src/main/java/io/opentelemetry/contrib/eventbridge/EventToSpanEventBridge.java

+ *   <li>{@link LogRecordData#getObservedTimestampEpochNanos()} is mapped to span event attribute
+ *       with key {@code log.record.observed_timestamp}
+ *   <li>{@link LogRecordData#getSeverity()} is mapped to span event attribute with key {@code
+ *       log.record.severity_number}


Do you like log.record.severity_number better than simply log.record.severity? Maybe you're just trying to maintain consistency with the protos?

Maybe you're just trying to maintain consistency with the protos?

Exactly... if / when the spec comes out and tell us how to translate between events and span events, we'll of course follow those instructions. But until then, the less opinionated we can be the better, and following the proto field naming is a way to avoid having an opinion.

https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/logs/v1/logs.proto#L161

breedx-splk · 2024-11-18T21:41:45Z

processors/src/main/java/io/opentelemetry/contrib/eventbridge/EventToSpanEventBridge.java

+ */
+public final class EventToSpanEventBridge implements LogRecordProcessor {
+
+  private static final Logger LOGGER = Logger.getLogger(EventToSpanEventBridge.class.getName());


We use lowercase for these in almost every other place. Also, I know it's inconsistent with the convention of naming static final constants all uppercase....conflicting conventions are at play here.

Suggested change

private static final Logger LOGGER = Logger.getLogger(EventToSpanEventBridge.class.getName());

private static final Logger logger = Logger.getLogger(EventToSpanEventBridge.class.getName());

breedx-splk · 2024-11-18T21:43:29Z

processors/src/main/java/io/opentelemetry/contrib/eventbridge/EventToSpanEventBridge.java

+    Value<?> body = logRecord.getBodyValue();
+    if (body != null) {
+      MarshalerWithSize marshaler = AnyValueMarshaler.create(body);
+      ByteArrayOutputStream baos = new ByteArrayOutputStream();


I'd prefer to use the more general type on the left-hand side and then the variable name also gets less cryptic.

Suggested change

ByteArrayOutputStream baos = new ByteArrayOutputStream();

OutputStream out = new ByteArrayOutputStream();

breedx-splk · 2024-11-18T21:44:15Z

.github/component_owners.yml

@@ -56,6 +56,7 @@ components:
  processors:
    - LikeTheSalad
    - breedx-splk
+    - jack-berg


Thanks for the help!

breedx-splk · 2024-11-18T21:50:03Z

processors/src/main/java/io/opentelemetry/contrib/eventbridge/EventToSpanEventBridge.java

+    LogRecordData logRecordData = logRecord.toLogRecordData();
+    String eventName = logRecordData.getAttributes().get(EVENT_NAME);
+    if (eventName == null) {
+      return;
+    }


Doesn't have to be addressed in this PR, but it's a little disappointing that we have to do the toLogRecordData() conversion first in order to check to see if an attribute exists on the ReadWriteLogRecord. In fact, the ReadWriteLogRecord doesn't allow much reading at all, does it...? /rant

At least we're within onEmit which should only be relevant for events (so that event.name is never actually null) but still... :)

It is disappointing and needlessly wasteful. We have a todo to add more getters for directly accessing bits of info in ReadWriteLogRecord. May be time to pick that up..

Separately, I wonder if we can / should add a helper method boolean isEvent() since we anticipate this being a common thing to access.

Yeah, I like that idea. We can revisit this here later when that happens.

breedx-splk · 2024-11-18T22:03:26Z

processors/src/test/java/io/opentelemetry/contrib/eventbridge/EventToSpanEventBridgeTest.java

+      span.end();
+    }
+
+    assertThat(spanExporter.getFinishedSpanItems()).isEmpty();


It feels like the goal was to test the isRecording() check/functionality but instead it seems like this is just testing out the sampler. Maybe it's the case that the SamplingResult determines what the implementation of isRecording() returns? In any case, it's not super obvious to the reader why we would be testing that sampling works here.

Maybe it's the case that the SamplingResult determines what the implementation of isRecording() returns?

Yes that's right. I can add a comment to help the reader connect these dots.

Add EventToSpanEventBridge

20f4ddd

jack-berg requested a review from a team as a code owner November 18, 2024 19:42

github-actions bot assigned breedx-splk, jack-berg, LikeTheSalad and trask Nov 18, 2024

github-actions bot requested review from breedx-splk, LikeTheSalad and trask November 18, 2024 19:42

This was referenced Nov 18, 2024

Demonstrate span-event-to-event, event-to-span-event bridging open-telemetry/opentelemetry-java#6650

Closed

Option to convert log events to span events open-telemetry/opentelemetry-java#6880

Open

jack-berg added 3 commits November 18, 2024 15:33

Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…

02d0789

…y-java-contrib into event-to-span-event-bridge

Fix build, add jack-berg to processors component owners

6dcedc0

fix link

3b2e9d5

breedx-splk reviewed Nov 18, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add LogRecordProcessor to record event log records as span events #1551

Add LogRecordProcessor to record event log records as span events #1551

jack-berg commented Nov 18, 2024

jack-berg commented Nov 18, 2024

breedx-splk left a comment

breedx-splk Nov 18, 2024

breedx-splk Nov 18, 2024

jack-berg Nov 18, 2024

breedx-splk Nov 18, 2024

breedx-splk Nov 18, 2024

jack-berg Nov 18, 2024

breedx-splk Nov 18, 2024

breedx-splk Nov 18, 2024

breedx-splk Nov 18, 2024

breedx-splk Nov 18, 2024

jack-berg Nov 18, 2024

breedx-splk Nov 18, 2024

breedx-splk Nov 18, 2024

jack-berg Nov 18, 2024

	private static final Logger LOGGER = Logger.getLogger(EventToSpanEventBridge.class.getName());
	private static final Logger logger = Logger.getLogger(EventToSpanEventBridge.class.getName());

	ByteArrayOutputStream baos = new ByteArrayOutputStream();
	OutputStream out = new ByteArrayOutputStream();

Add LogRecordProcessor to record event log records as span events #1551

Are you sure you want to change the base?

Add LogRecordProcessor to record event log records as span events #1551

Conversation

jack-berg commented Nov 18, 2024

jack-berg commented Nov 18, 2024

breedx-splk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment