Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] QueryGroup Resource Tracking framework and implementation (#13897) #15154

Conversation

kiranprakash154
Copy link
Contributor

Backport - #13897

…-project#13897)

* initial code for the sandbox resource tracking and cancellation framework

Signed-off-by: Kiran Prakash <[email protected]>

* Fix Failing Tests

Signed-off-by: Kiran Prakash <[email protected]>

* spotless Apply

Signed-off-by: Kiran Prakash <[email protected]>

* Update SandboxService.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update SandboxService.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update SandboxTask.java

Signed-off-by: Kiran Prakash <[email protected]>

* Add java docs

Signed-off-by: Kiran Prakash <[email protected]>

* spotless

Signed-off-by: Kiran Prakash <[email protected]>

* javadocs

Signed-off-by: Kiran Prakash <[email protected]>

* javadocs

Signed-off-by: Kiran Prakash <[email protected]>

* java docs

Signed-off-by: Kiran Prakash <[email protected]>

* Update AbstractTaskCancellation.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update SandboxModule.java

Signed-off-by: Kiran Prakash <[email protected]>

* Some tests and stubs

Signed-off-by: Kiran Prakash <[email protected]>

* spotless

Signed-off-by: Kiran Prakash <[email protected]>

* :server:testingConventions

Signed-off-by: Kiran Prakash <[email protected]>

* Update AbstractTaskCancellation.java

Signed-off-by: Kiran Prakash <[email protected]>

* more tests

Signed-off-by: Kiran Prakash <[email protected]>

* addressing comments

Signed-off-by: Kiran Prakash <[email protected]>

* revert some accidentally pushed files

Signed-off-by: Kiran Prakash <[email protected]>

* resolve flakiness

Signed-off-by: Kiran Prakash <[email protected]>

* renaming sandbox to querygroup and adjusting code based on merged PRs

Signed-off-by: Kiran Prakash <[email protected]>

* jvm to memory

Signed-off-by: Kiran Prakash <[email protected]>

* missing java docs

Signed-off-by: Kiran Prakash <[email protected]>

* spotless

Signed-off-by: Kiran Prakash <[email protected]>

* Update CHANGELOG.md
Signed-off-by: Kiran Prakash <[email protected]>

* pluck cancellation changes out of this PR

Signed-off-by: Kiran Prakash <[email protected]>

* remove unused

Signed-off-by: Kiran Prakash <[email protected]>

* remove cancellation related code and add more tests coverage

Signed-off-by: Kiran Prakash <[email protected]>

* us only memory and not jvm

Signed-off-by: Kiran Prakash <[email protected]>

* test conventions

Signed-off-by: Kiran Prakash <[email protected]>

* Bring back enum

Signed-off-by: Kiran Prakash <[email protected]>

* Update SearchBackpressureService.java

Signed-off-by: Kiran Prakash <[email protected]>

* revert changes

Signed-off-by: Kiran Prakash <[email protected]>

* revert changes

Signed-off-by: Kiran Prakash <[email protected]>

* all required changes

Signed-off-by: Kiran Prakash <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Kiran Prakash <[email protected]>

* cleanups

Signed-off-by: Kiran Prakash <[email protected]>

* Delete QueryGroupService.java

Signed-off-by: Kiran Prakash <[email protected]>

* cleanups

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupLevelResourceUsageViewTests.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupLevelResourceUsageViewTests.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupResourceUsageTrackerService.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupResourceUsageTrackerService.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupResourceUsageTrackerService.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update CHANGELOG.md
Signed-off-by: Kiran Prakash <[email protected]>

* rebasing with latest main

Signed-off-by: Kiran Prakash <[email protected]>

* remove experimental

Signed-off-by: Kiran Prakash <[email protected]>

* remove queryGroupId

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupResourceUsageTrackerService.java

Signed-off-by: Kiran Prakash <[email protected]>

* change code comments

Signed-off-by: Kiran Prakash <[email protected]>

* remmove QueryGroupUsageTracker

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupResourceUsageTrackerService.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupResourceUsageTrackerService.java

Signed-off-by: Kiran Prakash <[email protected]>

* remove QueryGroupTestHelpers

Signed-off-by: Kiran Prakash <[email protected]>

* cleanups

Signed-off-by: Kiran Prakash <[email protected]>

* remove queryGroupHelper

Signed-off-by: Kiran Prakash <[email protected]>

* Update ResourceTypeTests.java

Signed-off-by: Kiran Prakash <[email protected]>

* extend OpenSearchTestCase

Signed-off-by: Kiran Prakash <[email protected]>

* pr comments

Signed-off-by: Kiran Prakash <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Kiran Prakash <[email protected]>

* Update QueryGroupResourceUsageTrackerServiceTests.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update ResourceTypeTests.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update ResourceTypeTests.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update ResourceType.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update ResourceType.java

Signed-off-by: Kiran Prakash <[email protected]>

---------

Signed-off-by: Kiran Prakash <[email protected]>
(cherry picked from commit 97c1bf0)
@kiranprakash154 kiranprakash154 self-assigned this Aug 7, 2024
@kiranprakash154 kiranprakash154 changed the title QueryGroup Resource Tracking framework and implementation (#13897) [Backport 2.x] QueryGroup Resource Tracking framework and implementation (#13897) Aug 7, 2024
Copy link
Contributor

github-actions bot commented Aug 7, 2024

❕ Gradle check result for 0e81e4e: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testWriteBlobWithRetries

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Aug 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.61%. Comparing base (e5e1bad) to head (0e81e4e).
Report is 31 commits behind head on 2.x.

Additional details and impacted files
@@             Coverage Diff              @@
##                2.x   #15154      +/-   ##
============================================
+ Coverage     71.48%   71.61%   +0.13%     
- Complexity    62677    62780     +103     
============================================
  Files          5123     5129       +6     
  Lines        294480   294686     +206     
  Branches      42912    42941      +29     
============================================
+ Hits         210518   211053     +535     
+ Misses        66286    65951     -335     
- Partials      17676    17682       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kiranprakash154
Copy link
Contributor Author

@jed326 can u merge this ?
(cc: @jainankitk )

Copy link
Collaborator

@jainankitk jainankitk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't seem to have any conflicts!

@jed326 jed326 merged commit 1015fa6 into opensearch-project:2.x Aug 8, 2024
58 of 60 checks passed
@kiranprakash154 kiranprakash154 deleted the backport/backport-13897-to-2.x branch August 8, 2024 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants