You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For clusters with large number of shards and indices, the following computation is sub-optimal and leads to higher latencies for Rest APIs such as _cat/segments.
Create a cluster with high number of shards and indices (say 30k indices and 100k shards).
The latency of _cat/segments API will increase manifold.
Expected behavior
Refactoring using a map Map<String, List<ShardSegments>> indicesShardSegments = new HashMap<>(); will help save on extra computations and decrease latency of REST APIs.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
OS: [e.g. iOS]
Version [e.g. 22]
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
Thanks @gargharsh3134 for finding this and raising a draft PR as well. Please capture improvement in 1/ latency and 2/ heap usage so that we know the impact of this change.
Describe the bug
For clusters with large number of shards and indices, the following computation is sub-optimal and leads to higher latencies for Rest APIs such as _cat/segments.
https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/action/admin/indices/segments/IndicesSegmentResponse.java#L96
Related component
Cluster Manager
To Reproduce
Create a cluster with high number of shards and indices (say 30k indices and 100k shards).
The latency of
_cat/segments
API will increase manifold.Expected behavior
Refactoring using a map
Map<String, List<ShardSegments>> indicesShardSegments = new HashMap<>();
will help save on extra computations and decrease latency of REST APIs.Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: