Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] High number of unnecessary computations in getIndices() of IndicesSegmentResponse #14720

Closed
gargharsh3134 opened this issue Jul 11, 2024 · 2 comments · Fixed by #14717
Closed
Labels
bug Something isn't working Cluster Manager good first issue Good for newcomers

Comments

@gargharsh3134
Copy link
Contributor

Describe the bug

For clusters with large number of shards and indices, the following computation is sub-optimal and leads to higher latencies for Rest APIs such as _cat/segments.

https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/action/admin/indices/segments/IndicesSegmentResponse.java#L96

Related component

Cluster Manager

To Reproduce

Create a cluster with high number of shards and indices (say 30k indices and 100k shards).
The latency of _cat/segments API will increase manifold.

Expected behavior

Refactoring using a map Map<String, List<ShardSegments>> indicesShardSegments = new HashMap<>(); will help save on extra computations and decrease latency of REST APIs.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@dblock
Copy link
Member

dblock commented Jul 29, 2024

[Catch All Triage - 1, 2, 3, 4]

@sandeshkr419 sandeshkr419 added the good first issue Good for newcomers label Aug 9, 2024
@sandeshkr419
Copy link
Contributor

Thanks @gargharsh3134 for finding this and raising a draft PR as well. Please capture improvement in 1/ latency and 2/ heap usage so that we know the impact of this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Cluster Manager good first issue Good for newcomers
Projects
Status: ✅ Done
Development

Successfully merging a pull request may close this issue.

3 participants