review truncating otel strings that are indexed as keywords

[simitt]

We currently truncate otel attributes that are indexed as keywords to 1024 chars

apm-data/input/otlp/traces.go

Line 279 in 88a3977

stringval := truncate(v.Str())

The mappings are generally created with ignore_above: 1024, which would lead to not indexing this field if the value exceeds 1024 chars.

We should review if truncating of the values for otel strings is the best choice, where the field will always be indexed, but anything above 1024 chars will be completely lost vs. not truncating the values, leading to certain fields not being indexed and searchable, but only available in _source, if exceeding the limit. When moving to synthetic source, the time to retrieve the non-indexed values might be increased.

[richcollier]

@simitt - is there a way for the customer to easily override this? Or some other kind of workaround?

[simitt]

No this is unfortunately not possible currently.