Top N queries by Latency - aggregator implementation #11295
Labels
enhancement
Enhancement or improvement to existing feature or request
Search:Query Insights
untriaged
v2.12.0
Issues and PRs related to version 2.12.0
Is your feature request related to a problem? Please describe.
Currently, OpenSearch lacks a direct means of providing insights into “top queries” that have a significant impact on latency, we want to build the in-memory aggregator to capture those queries on the coordinator node level.
(Parent rfc: #11186)
Describe the solution you'd like
Implement a priority queue-based in-memory data store, with configurable window size, on the coordinator node, designed to efficiently store the top N queries. The data model of the stored query attributes should be highly extensible for different types of resources and metrics.
Describe alternatives you've considered
We can instead export the data and offload the query instrumentation logic to external sinks. But it would require us to retain all necessary data for every request. Additionally, as of now we lack a state-of-the-art on-node store capable of holding all request information for aggregation and calculating the top N when user requested. Moreover, we refrain from introducing dependencies on a specific sink in this feature.
Additional context
Please see parent rfc: #11186
The text was updated successfully, but these errors were encountered: