You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Doc level monitors intend to track sequence number of indices shards and query only documents ingested since last monitor execution. The problem is that users configure doc level monitors to run against index patterns with rollover or data streams.
More often than not doc level monitors are querying an index patter which resolves to , say 1000 concrete indices, of which 999 are read-only and 1 write index.
We really don't need to query all the 999 read-only indices as they might not have any documents ingested since the previous execution.
we can optimize to query only those indices which are writeable and all indices which have been marked as read-only AFTER the previous monitor execution for first run of monitor, when creating doc level queries it should only consider write index and the read indices which were created after the enabled_time of monitor
This will drastically reduce the CPU consumption of the doc level monitor and provide a significant optimization for the plugin's resource consumption
The text was updated successfully, but these errors were encountered:
Doc level monitors intend to track sequence number of indices shards and query only documents ingested since last monitor execution. The problem is that users configure doc level monitors to run against index patterns with rollover or data streams.
More often than not doc level monitors are querying an index patter which resolves to , say 1000 concrete indices, of which 999 are read-only and 1 write index.
We really don't need to query all the 999 read-only indices as they might not have any documents ingested since the previous execution.
we can optimize to query only those indices which are writeable and all indices which have been marked as read-only AFTER the previous monitor execution
for first run of monitor, when creating doc level queries it should only consider write index and the read indices which were created after the enabled_time of monitor
This will drastically reduce the CPU consumption of the doc level monitor and provide a significant optimization for the plugin's resource consumption
The text was updated successfully, but these errors were encountered: