[REQUEST]: [Fleet] document breaking change with custom pipeline naming in 8.12.1

[kpollich]

Description

I'm providing a draft of a breaking change note to include in the 8.12.1 release notes. Please feel free to make suggestions or iterate on this as needed. Many thanks! cc @kilfoyle

---

If you were relying on an ingest pipeline of the form ${type}-${integration}@custom introduced in 8.12.0 (e.g. traces-apm@custom or logs-nginx@custom or metrics-system@custom) you will need to update your pipeline's name to include a .integration suffix to preserve your ingestion behavior, e.g. logs-nginx.integration@custom.

In 8.12.0, Fleet added new custom ingest pipeline names for adding custom processing to integration data streams. These pipeline names used patterns as follows:

• global@custom
• ${type}@custom e.g. traces@custom
• ${type}-${integration}@custom e.g. traces-apm@custom
• ${type}-${integration}-${dataset}@custom pre-existing, e.g. traces-apm.rum@custom

However, it was discovered in elastic/kibana#175254 that the ${type-integration}@custom pattern can collide in cases where the integration name is also a dataset name. The clearest case of these collisions was in the APM integration's data streams, e.g.

• traces-apm
• traces-apm.rum
• traces-apm.sampled

Because traces-apm is a legitimate data stream defined by the APM integration here, it incurred a collision of these custom pipeline names on 8.12.0, e.g.

// traces-apm
{
  "pipeline": {
    "name": "traces-apm@custom", // <---
    "ignore_missing_pipeline": true
  }
}

// traces-apm.rum
{
  "pipeline": {
    "name": "traces-apm@custom", // <---
    "ignore_missing_pipeline": true
  }
},
{
  "pipeline": {
    "name": "traces-apm.rum@custom",
    "ignore_missing_pipeline": true
  }
}

Prior to 8.12.0, the traces-apm@custom custom pipeline name was already supported. So, if you had already defined and were using the supported traces-apm@custom pipeline, then upgrade to 8.12.0, you would observe that documents ingested to traces-apm.rum and traces-apm.sampled would also be processed by your preexisting traces-apm@custom ingest pipeline. This could cause breakages and unexpected pipeline processing errors.

To correct this in 8.12.1, Fleet will now append a suffix to the "integration level" custom ingest pipeline name. The new suffix will prevent collisions between datasets and integration names moving forward, e.g.

// traces-apm
{
  "pipeline": {
    "name": "traces-apm.integration@custom", // <--- Integration level pipeline
    "ignore_missing_pipeline": true
  }
},
{
  "pipeline": {
    "name": "traces-apm@custom", // <--- Dataset level pipeline
    "ignore_missing_pipeline": true
  }
}

// traces-apm.rum
{
  "pipeline": {
    "name": "traces-apm.integration@custom", // <--- Integration level pipeline
    "ignore_missing_pipeline": true
  }
},
{
  "pipeline": {
    "name": "traces-apm.rum@custom", // <--- Dataset level pipeline
    "ignore_missing_pipeline": true
  }
}

So, if you are relying on an integration level custom ingest pipeline introduced in 8.12.0, you will need to update its name to include the new .integration suffix to preserve your existing ingestion behavior.

Resources

• [Fleet] Potential breaking change with APM data streams (maybe others) and Fleet ingest pipeline customization hooks kibana#175254
• [Fleet] Update Fleet's custom ingest pipeline names to avoid collisions + add descriptions to each pipeline kibana#175448
• [Fleet] Allow to have custom pipeline at different level kibana#170270

Collaboration

The product team will provide the initial content.

Point of contact.

Main contact: @kpollich

Stakeholders: @elastic/fleet