[FLAKY ON MKI] x-pack/test_serverless/functional/test_suites/observability/dataset_quality/degraded_field_flyout.ts

[wayneseymour]

Summary

This test started to fail consistently (in 3 of the last 3 runs)

Test

serverless observability UI Dataset Quality Degraded fields flyout testing root cause for ignored fields current field limit issues should display increase field limit as a possible mitigation for special packages like apm app

Error message

Error: expected testSubject(datasetQualityDetailsDegradedFieldFlyoutFieldLimitMitigationAccordion) to exist
    at TestSubjects.existOrFail (test_subjects.ts:76:13)
    at Context. (degraded_field_flyout.ts:822:11)
    at Object.apply (wrap_function.js:74:16)

Screenshot

Notes

Here's the Kibana commit diff between the last successful and the first failing run: 08c3bc2...6c0cf0e
This PR from the list looks related: #198692

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[weltenwort]

Looks like I finally found the difference between the environments that causes this: Locally and in CI the geoip ingest pipeline processor adds a field: tags: "_geoip_database_unavailable_GeoLite2-City.mmdb" to every document ingested into the APM app log data streams, but on MKI it does not. Since the test is intentionally trying to push the number of fields over the limit to trigger the ignore_dynamic_beyond_limit index setting, this breaks the test.

To me it sounds like the proper solution would be to make sure the Elasticsearch geoip processor works correctly in the local and CI serverless test environment as well and to align the data quality tests with that expectation.