ECS Version to 8.x

[uberjew666]

This there any roadmap or plan to upgrade the ECS version from 1.2 to 8.x?

[robsonhermes]

I have a question about this. The ecs.version seems to be set to 1.2.0.
Does this version refer to the ECS Layout?

What does it mean to developers using this encoder? We would not be able to log fields that were added in later versions?
We have Java and Ruby systems. The Ruby encoder, in the other hand, currently is set to ECS version 1.4.0. What could happen if we have java services logging with 1.2.0 layout and ruby services with 1.4.0?

Could this old version somehow limit what we could do with these logs in Kibana? The reason I ask is that is because currently I'm using Logstash encoder and plan to imgrate to ECS encoder. I then wonder if this version 1.2.0 could result if any sort of limitation in terms of what we could do with the logs given the current version of ECS layout is 8.16.

Hopefully that makes sense.

[JonasKunz]

What does it mean to developers using this encoder? We would not be able to log fields that were added in later versions?

No, the ecs.version doesn't have any impact on the log documents this lib produces:
All the fields logged by default (e.g. log.level) have had no breaking changes to them since 1.2.0, so the documents are als conforming to the 8.16.0 standard. At the same we don't do any validation of custom fields provided bia the MDC or the additional-fields config options: There, you are yourself responsible to ensure that it conforms to the ECS version you are targeting. However, the library will output whatever you give to without objections.

Could this old version somehow limit what we could do with these logs in Kibana? The reason I ask is that is because currently I'm using Logstash encoder and plan to imgrate to ECS encoder. I then wonder if this version 1.2.0 could result if any sort of limitation in terms of what we could do with the logs given the current version of ECS layout is 8.16.

I'm not a kibana expert, but the docs on ecs.version state the following:

When querying across multiple indices — which may conform to slightly different ECS versions — this field lets integrations adjust to the schema version of the events.

So this field serves just for enabling Kibana integrations to be built in a backwards compatible manner: E.g. if a field was renamed from one ECS version to another, kibana integrations can use the ecs.version to still correctly interpret the document. I'm not aware of any concrete examples of problems in this regard with this ecs-logging library though.

I'll double check my comment here with my colleagues to see if I'm missing something.

This there any roadmap or plan to upgrade the ECS version from 1.2 to 8.x?

I think what we could do is allow users to customize the value of ecs.version field, e.g. if they want to add some field via additional-fields which is conforming to a more recent ECS version and want it to be guaranteed to be picked up correctly by kibana integrations.

[uberjew666]

I like the idea of allowing users the ability to set the ecs.version field. It's caused a bit of confusion between our team and others since Logstash appends a 8.x version tag to the final event. Having these match would help immensely.

Thank you for your response @JonasKunz