Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memoize isOnRemoteNode in index settings and refactor as well #12994

Merged
merged 2 commits into from
Apr 2, 2024

Conversation

gbbafna
Copy link
Collaborator

@gbbafna gbbafna commented Apr 1, 2024

Description

Memoize isOnRemoteNode in index settings and refactor as well

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

github-actions bot commented Apr 1, 2024

Compatibility status:

Checks if related components are compatible with change 6b86f24

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/performance-analyzer.git]

Copy link
Contributor

github-actions bot commented Apr 1, 2024

✅ Gradle check result for bab1201: SUCCESS

Copy link

codecov bot commented Apr 1, 2024

Codecov Report

Attention: Patch coverage is 36.36364% with 14 lines in your changes are missing coverage. Please review.

Project coverage is 71.37%. Comparing base (b15cb0c) to head (6b86f24).
Report is 108 commits behind head on main.

Files Patch % Lines
...in/java/org/opensearch/index/shard/IndexShard.java 30.76% 2 Missing and 7 partials ⚠️
.../opensearch/index/engine/NRTReplicationEngine.java 0.00% 0 Missing and 1 partial ⚠️
...h/index/remote/RemoteStoreStatsTrackerFactory.java 0.00% 0 Missing and 1 partial ⚠️
...rc/main/java/org/opensearch/index/store/Store.java 0.00% 0 Missing and 1 partial ⚠️
...in/java/org/opensearch/indices/IndicesService.java 0.00% 0 Missing and 1 partial ⚠️
...ch/indices/recovery/PeerRecoveryTargetService.java 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #12994      +/-   ##
============================================
- Coverage     71.42%   71.37%   -0.05%     
- Complexity    59978    60304     +326     
============================================
  Files          4985     5021      +36     
  Lines        282275   284068    +1793     
  Branches      40946    41151     +205     
============================================
+ Hits         201603   202750    +1147     
- Misses        63999    64465     +466     
- Partials      16673    16853     +180     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Gaurav Bafna <[email protected]>
Copy link
Contributor

github-actions bot commented Apr 1, 2024

✅ Gradle check result for 6b86f24: SUCCESS

Copy link
Contributor

@stephen-crawford stephen-crawford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @gbbafna,

Thanks for putting this together.

The changes all look pretty straightforward but I am not sure how much impact this caching will have. I see how you are checking whether the settings is present on the remote node but I don't see where that logic is used.

Since you swapped

 public boolean isRemoteNode() {
        return RemoteStoreNodeAttribute.isRemoteDataAttributePresent(this.getNodeSettings());

for

public boolean isAssignedOnRemoteNode() {
        return assignedOnRemoteNode;

It seems like the change of

return ReplicationType.SEGMENT.equals(replicationType) || isRemoteNode();

for

return ReplicationType.SEGMENT.equals(replicationType) || isAssignedOnRemoteNode();

maintains the same behavior but is reading from the variable. Presumably this would show some speed up but it is not clear to me how much of a difference this would make. Is this the extent of the intended speed up or are there further changes you had in mind?

Do you have any testing to compare the results with/without the change?

Copy link
Member

@ashking94 ashking94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this should reduce the unnecessary look ups as well.

@gbbafna
Copy link
Collaborator Author

gbbafna commented Apr 2, 2024

Is this the extent of the intended speed up or are there further changes you had in mind?

I don't have extent of the intended speed up . This is because remote store migration is a new feature under development and is experimental . We have reduced the unnecessary map look ups , similar to other index settings as well .

@sachinpkale sachinpkale merged commit eb66f62 into opensearch-project:main Apr 2, 2024
31 of 34 checks passed
@sachinpkale sachinpkale added the backport 2.x Backport to 2.x branch label Apr 2, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Apr 2, 2024
* Memoize isOnRemoteNode in index settings and rename it as well
---------
Signed-off-by: Gaurav Bafna <[email protected]>
(cherry picked from commit eb66f62)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gbbafna pushed a commit that referenced this pull request Apr 3, 2024
#13016)


Signed-off-by: Gaurav Bafna <[email protected]>
(cherry picked from commit eb66f62)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…arch-project#12994)

* Memoize isOnRemoteNode in index settings and rename it as well
---------
Signed-off-by: Gaurav Bafna <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
harshavamsi pushed a commit to harshavamsi/OpenSearch that referenced this pull request Apr 29, 2024
…arch-project#12994)

* Memoize isOnRemoteNode in index settings and rename it as well
---------
Signed-off-by: Gaurav Bafna <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants