[Backport 2.x] [Remote Store] Fix refresh lag bug on primary term change

[opensearch-trigger-bot]

Backport b17d4a8 from #10918.

[github-actions]

Compatibility status:

Checks if related components are compatible with change d07d558

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git]

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/29045/
• CommitID: d07d558
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      2 org.opensearch.index.shard.RemoteIndexShardTests.classMethod
      1 org.opensearch.index.shard.RemoteIndexShardTests.testSegRepSucceedsOnPreviousCopiedFiles

• URL: https://build.ci.opensearch.org/job/gradle-check/29048/
• CommitID: d07d558
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Merging #10935 (d07d558) into 2.x (1d92801) will increase coverage by 0.05%.
Report is 7 commits behind head on 2.x.
The diff coverage is 52.69%.

@@             Coverage Diff              @@
##                2.x   #10935      +/-   ##
============================================
+ Coverage     70.91%   70.96%   +0.05%     
- Complexity    58802    58809       +7     
============================================
  Files          4844     4847       +3     
  Lines        277512   277638     +126     
  Branches      40708    40721      +13     
============================================
+ Hits         196802   197033     +231     
+ Misses        64036    63816     -220     
- Partials      16674    16789     +115     

Files	Coverage Δ
...search/cluster/coordination/CoordinationState.java	88.39% <ø> (ø)
.../org/opensearch/cluster/service/MasterService.java	84.51% <100.00%> (-0.56%)	⬇️
...nsearch/gateway/remote/RemotePersistenceStats.java	100.00% <100.00%> (ø)
...in/java/org/opensearch/index/shard/IndexShard.java	69.78% <100.00%> (+0.58%)	⬆️
...earch/index/store/RemoteSegmentStoreDirectory.java	88.84% <ø> (-0.04%)	⬇️
...rc/main/java/org/opensearch/index/store/Store.java	80.68% <100.00%> (-0.18%)	⬇️
...h/cluster/coordination/InMemoryPersistedState.java	68.75% <0.00%> (-4.59%)	⬇️
...search/index/shard/RemoteStoreRefreshListener.java	86.82% <87.50%> (-0.30%)	⬇️
...g/opensearch/cluster/coordination/Coordinator.java	78.57% <77.77%> (+3.43%)	⬆️
.../java/org/opensearch/gateway/GatewayMetaState.java	71.60% <77.77%> (+2.35%)	⬆️
... and 4 more

... and 466 files with indirect coverage changes