Warn on too many fields in data view

[flash1293]

"Mapping explosion" (having indices or index patterns with a very large number of fields) is a known problem in the Elastic stack which is already mitigated in some ways on the Elasticsearch side. However, it's still possible to run into this case.

In most situations, a mapping explosion is not breaking a cluster or any functionality - instead, it is slowing down all operations, sometimes considerably.

Kibana is also vulnerable to this - as it's loading the full field list via _field_caps API for a configured data view both on Kibana server and browser client, a large number of fields can cause performance issues like sluggish UI / high CPU usage in the browser or event loop delays on Kibana server.

In these situations, the user often doesn't realize the underlying root cause is a mapping explosion - it just feels like a badly performing system in general which leaves users frustrated.

Currently, there is no mitigation for this problem happening on the Kibana side. To help users realizing and fixing the issue as quickly as possible, a quick win would be to create a dedicated documentation page for the problem and to warn about data views with a number of fields above a certain threshold (e.g. 10k) via warning toast (linking to the documentation page) which can be muted by the user.

cc @sixstringcode @mattkime @rudolf

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServicesSv)

[rudolf]

In addition to notifications in the UI, I think it would also be helpful to have server logs which could help users, and especially Elastic support more easily pinpoint the problem.

Taking one step further I think it would also make sense to have "circuit breakers" to protect Kibana's performance:

1. Would it make sense to stop processing a _field_caps response as soon as we realize it has more than a certain amount of fields? This would depend on why this is such a CPU hog... if the majority of the CPU time is spent in decompressing/deserializing the payload then this won't help. But if a significant amount of time is spent manipulating/looping over the response data this could be a viable circuit breaker.
2. We could also configure the ES client to not deserialize/decompress very large response bodies with maxResponseSize Add support for maxResponseSize and maxCompressedResponseSize elasticsearch-js#1551 This "circuit breaker" would probably have to be configurable so that users can override it if needed.

[flash1293]

On a related note: Another general approach would be to remove this being a bottleneck - by selectively and lazily loading field lists instead of a full scan up front. This would help keeping the UI more responsive in a bunch of other places, too (right now the UI isn't showing up until the full field list is loaded), but is a much larger change and would require significant refactoring. I think it makes sense to go with the mitigation approach in the near term (warning toast, circuit breaker) and think about lazy field lists as a long-term tech debt project - it would also fit in nicely with support for flattened fields (as these can't possibly be fully loaded up front). cc @thomasneirynck

[thomasneirynck]

I think it makes sense to go with the mitigation approach in the near term (warning toast, circuit breaker)

ok, makes sense

but is a much larger change and would require significant refactoring ... think about lazy field lists as a long-term tech debt project

Could you explain a little more what you mean by lazy loading?

Do you mean limiting the _field_caps call to just the fields required to bootstrap. So e.g. The call to ES would look something like http://localhost:9200?kibana_sample_logs/_field_caps?fields=machine.os.keyword,cpu if the client (e.g. a Lens on a Dashboard) only requiresmachine.os,.keyword and cpu fields?

Similarly, are you thinking about caching field-list info in the client-SO? e.g. a Lens visualization which only requires geo.dest and cpu could cache the required field-caps in the Lens-SO.

Lens is just an example, the same would apply to Maps.

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[kertal]

@jughosta and @kertal synced about it, and we would suggest a banner / callout in the data view management, which is displayed, when the number of fields > X (X needs to be determined) .... we could sync with Elasticsearch team to get a good number of fields

[kertal]

Closing this because it's not planned to be resolved in the foreseeable future. It will be tracked in our Icebox and will be re-opened if our priorities change. Feel free to re-open if you think it should be melted sooner.