Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate IO Based AdmissionController to AdmissionControl Framework #12583

Merged
merged 4 commits into from
Mar 14, 2024

Conversation

ajaymovva
Copy link
Contributor

@ajaymovva ajaymovva commented Mar 11, 2024

Description

Integrated IO Based AdmissionController to AdmissionControl Framework.

Node Stats Output On Linux

"admission_control": {
    "global_io_usage": {
        "transport": {
            "rejection_count": {
                "search": 8
            }
        }
    },
    "global_cpu_usage": {
        "transport": {
            "rejection_count": {}
        }
    }
}

Node Stats Output On Mac

"admission_control": {
    "global_cpu_usage": {
        "transport": {
            "rejection_count": {
                "search": 4
            }
        }
    }
}

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@ajaymovva ajaymovva force-pushed the feature/io-based-ac branch from e551547 to b0f31ea Compare March 11, 2024 13:42
Copy link
Contributor

github-actions bot commented Mar 11, 2024

Compatibility status:

Checks if related components are compatible with change 4626c01

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer.git]

Copy link
Contributor

❌ Gradle check result for e551547: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for b0f31ea: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ajaymovva ajaymovva force-pushed the feature/io-based-ac branch from 93ee481 to 7a0519c Compare March 11, 2024 16:02
Copy link
Contributor

❌ Gradle check result for 93ee481: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 7a0519c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ajaymovva ajaymovva force-pushed the feature/io-based-ac branch from 7a0519c to d6c1b85 Compare March 11, 2024 16:38
Copy link
Contributor

❕ Gradle check result for d6c1b85: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.search.SearchWeightedRoutingIT.testMultiGetWithNetworkDisruption_FailOpenEnabled

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Mar 11, 2024

Codecov Report

Attention: Patch coverage is 83.60656% with 10 lines in your changes are missing coverage. Please review.

Project coverage is 71.33%. Comparing base (b15cb0c) to head (4626c01).
Report is 24 commits behind head on main.

Files Patch % Lines
...ontrol/controllers/IoBasedAdmissionController.java 81.81% 3 Missing and 3 partials ⚠️
...ting/admissioncontrol/AdmissionControlService.java 60.00% 0 Missing and 2 partials ⚠️
...ode/resource/tracker/NodeResourceUsageTracker.java 50.00% 0 Missing and 1 partial ⚠️
...l/settings/IoBasedAdmissionControllerSettings.java 95.23% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #12583      +/-   ##
============================================
- Coverage     71.42%   71.33%   -0.09%     
- Complexity    59978    60017      +39     
============================================
  Files          4985     4988       +3     
  Lines        282275   282490     +215     
  Branches      40946    40980      +34     
============================================
- Hits         201603   201504      -99     
- Misses        63999    64253     +254     
- Partials      16673    16733      +60     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ajaymovva added a commit to ajaymovva/OpenSearch that referenced this pull request Mar 14, 2024
…pensearch-project#12583)

* Integrated IO Based AdmissionController to AdmissionControl Framework

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Integration Tests Fixes to Add Delay for Trackers Update

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Fixing Integ Test and Removed Multi Node Tests

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Addressing Comments

Signed-off-by: Ajay Kumar Movva <[email protected]>

---------

Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
ajaymovva added a commit to ajaymovva/OpenSearch that referenced this pull request Mar 14, 2024
gbbafna pushed a commit that referenced this pull request Mar 14, 2024
andrross added a commit to andrross/OpenSearch that referenced this pull request Mar 14, 2024
…amework (opensearch-project#12583)"

This reverts commit b6b16d8.

Reverting as this introduced test failures detailed in opensearch-project#12664.
andrross added a commit to andrross/OpenSearch that referenced this pull request Mar 14, 2024
…amework (opensearch-project#12583)"

This reverts commit b6b16d8.

Reverting as this introduced test failures detailed in opensearch-project#12664.

Signed-off-by: Andrew Ross <[email protected]>
@@ -19,6 +19,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Remote reindex: Add support for configurable retry mechanism ([#12561](https://github.com/opensearch-project/OpenSearch/pull/12561))
- [Admission Control] Integrate IO Usage Tracker to the Resource Usage Collector Service and Emit IO Usage Stats ([#11880](https://github.com/opensearch-project/OpenSearch/pull/11880))
- Tracing for deep search path ([#12103](https://github.com/opensearch-project/OpenSearch/pull/12103))
- [Admission Control] Integrated IO Based AdmissionController to AdmissionControl Framework ([#12583](https://github.com/opensearch-project/OpenSearch/pull/12583))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, your changelog entry is in the wrong section, hence the cherry-pick failure on the backport.

andrross added a commit to andrross/OpenSearch that referenced this pull request Mar 14, 2024
…amework (opensearch-project#12583) (opensearch-project#12659)"

This reverts commit 3970ec4.

Reverting as this introduced test failures detailed in opensearch-project#12664.

Signed-off-by: Andrew Ross <[email protected]>
reta pushed a commit that referenced this pull request Mar 14, 2024
…amework (#12583)" (#12670)

This reverts commit b6b16d8.

Reverting as this introduced test failures detailed in #12664.

Signed-off-by: Andrew Ross <[email protected]>
reta pushed a commit that referenced this pull request Mar 14, 2024
…amework (#12583) (#12659)" (#12671)

This reverts commit 3970ec4.

Reverting as this introduced test failures detailed in #12664.

Signed-off-by: Andrew Ross <[email protected]>
ajaymovva added a commit to ajaymovva/OpenSearch that referenced this pull request Mar 16, 2024
…pensearch-project#12583)

* Integrated IO Based AdmissionController to AdmissionControl Framework

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Integration Tests Fixes to Add Delay for Trackers Update

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Fixing Integ Test and Removed Multi Node Tests

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Addressing Comments

Signed-off-by: Ajay Kumar Movva <[email protected]>

---------

Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
ajaymovva added a commit to ajaymovva/OpenSearch that referenced this pull request Mar 16, 2024
…pensearch-project#12583)

* Integrated IO Based AdmissionController to AdmissionControl Framework

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Integration Tests Fixes to Add Delay for Trackers Update

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Fixing Integ Test and Removed Multi Node Tests

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Addressing Comments

Signed-off-by: Ajay Kumar Movva <[email protected]>

---------

Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
ajaymovva added a commit to ajaymovva/OpenSearch that referenced this pull request Mar 17, 2024
…pensearch-project#12583)

* Integrated IO Based AdmissionController to AdmissionControl Framework

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Integration Tests Fixes to Add Delay for Trackers Update

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Fixing Integ Test and Removed Multi Node Tests

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Addressing Comments

Signed-off-by: Ajay Kumar Movva <[email protected]>

---------

Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
ajaymovva added a commit to ajaymovva/OpenSearch that referenced this pull request Mar 18, 2024
…pensearch-project#12583)

* Integrated IO Based AdmissionController to AdmissionControl Framework

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Integration Tests Fixes to Add Delay for Trackers Update

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Fixing Integ Test and Removed Multi Node Tests

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Addressing Comments

Signed-off-by: Ajay Kumar Movva <[email protected]>

---------

Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
sachinpkale pushed a commit that referenced this pull request Mar 18, 2024
…12702)

* Integrate IO Based AdmissionController to AdmissionControl Framework (#12583)
---------
Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
ajaymovva added a commit to ajaymovva/OpenSearch that referenced this pull request Mar 18, 2024
…pensearch-project#12702)

* Integrate IO Based AdmissionController to AdmissionControl Framework (opensearch-project#12583)
---------
Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
…pensearch-project#12583)

* Integrated IO Based AdmissionController to AdmissionControl Framework

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Integration Tests Fixes to Add Delay for Trackers Update

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Fixing Integ Test and Removed Multi Node Tests

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Addressing Comments

Signed-off-by: Ajay Kumar Movva <[email protected]>

---------

Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
…amework (opensearch-project#12583)" (opensearch-project#12670)

This reverts commit b6b16d8.

Reverting as this introduced test failures detailed in opensearch-project#12664.

Signed-off-by: Andrew Ross <[email protected]>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
…pensearch-project#12702)

* Integrate IO Based AdmissionController to AdmissionControl Framework (opensearch-project#12583)
---------
Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…pensearch-project#12583)

* Integrated IO Based AdmissionController to AdmissionControl Framework

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Integration Tests Fixes to Add Delay for Trackers Update

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Fixing Integ Test and Removed Multi Node Tests

Signed-off-by: Ajay Kumar Movva <[email protected]>

* Addressing Comments

Signed-off-by: Ajay Kumar Movva <[email protected]>

---------

Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…amework (opensearch-project#12583)" (opensearch-project#12670)

This reverts commit b6b16d8.

Reverting as this introduced test failures detailed in opensearch-project#12664.

Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…pensearch-project#12702)

* Integrate IO Based AdmissionController to AdmissionControl Framework (opensearch-project#12583)
---------
Signed-off-by: Ajay Kumar Movva <[email protected]>
Co-authored-by: Ajay Kumar Movva <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport-failed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants