Skip to content

Commit

Permalink
Added trace peer forwarder doc (opensearch-project#2245)
Browse files Browse the repository at this point in the history
* Added trace peer forwarder doc

Signed-off-by: Asif Sohail Mohammed <[email protected]>
  • Loading branch information
asifsmohammed authored Feb 9, 2023
1 parent 7ce7818 commit e32e1c9
Showing 1 changed file with 43 additions and 0 deletions.
43 changes: 43 additions & 0 deletions data-prepper-plugins/trace-peer-forwarder-processor/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Trace Peer Forwarder Processor

This processor is used to reduce the number of Events that will be forwarded in a Trace Analytics pipeline by half when using [Peer Forwarder](https://github.com/opensearch-project/data-prepper/blob/main/docs/peer_forwarder.md).
It groups the events based on `trace_id` similar to `service_map_stateful` and `otel_trace_raw ` processors.

In [Trace Analytics pipeline](https://github.com/opensearch-project/data-prepper/blob/main/docs/trace_analytics.md#trace-analytics-pipeline) each event is duplicated, when it is sent from `otel-trace-pipeline` to `raw-pipeline` and `service-map-pipeline`.
So, the event will be forwarded once in each pipeline. Using this processor event will be forwarded only once in `otel-trace-pipeline` to correct peer.

## Basic Usage
To get started with trace peer forwarder, create the following `pipeline.yaml` along with [Peer Forwarder](https://github.com/opensearch-project/data-prepper/blob/main/docs/peer_forwarder.md) in `data-prepper-config.yaml`

```yaml
otel-trace-pipeline:
delay: "100"
source:
otel_trace_source:
processor:
- trace_peer_forwarder:
sink:
- pipeline:
name: "raw-pipeline"
- pipeline:
name: "service-map-pipeline"
raw-pipeline:
source:
pipeline:
name: "entry-pipeline"
processor:
- otel_trace_raw:
sink:
- opensearch:
service-map-pipeline:
delay: "100"
source:
pipeline:
name: "entry-pipeline"
processor:
- service_map_stateful:
sink:
- opensearch:
```
In the above pipeline, events will be forwarded in the `otel-trace-pipeline` to correct peer and no forwarding will be done in `raw-pipeline` or `service-map-pipeline`.

0 comments on commit e32e1c9

Please sign in to comment.