Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tiered Caching] Indices Request cache stalekey management #12625

Merged
merged 29 commits into from
Mar 17, 2024
Merged

[Tiered Caching] Indices Request cache stalekey management #12625

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
94e12c5
Introduce IndicesRequestCacheCleanupManager
kiranprakash154 Mar 12, 2024
78190da
using cleanup mgr to enqueue cleanups
kiranprakash154 Mar 12, 2024
a4ed23e
readability improvements
kiranprakash154 Mar 12, 2024
dcf5dc1
update updateCleanupKeyToCountMap on new cache entry
kiranprakash154 Mar 12, 2024
7ecad50
create IndicesRequestCacheCleanupManager & settings and validators
kiranprakash154 Mar 12, 2024
5d32025
Add IRC specific cache cleaner and remove from IndicesService
kiranprakash154 Mar 12, 2024
534f603
Unit Tests
kiranprakash154 Mar 12, 2024
8b0ccb9
Update CHANGELOG.md
kiranprakash154 Mar 12, 2024
6975e0c
move cachecleaner inside mgr
kiranprakash154 Mar 13, 2024
2deb001
remove processCleanupKeys
kiranprakash154 Mar 13, 2024
7b81cb9
minor cleanups
kiranprakash154 Mar 13, 2024
840f5c3
add updateCleanupKeyToCountMapOnCacheEviction
kiranprakash154 Mar 13, 2024
1de2f66
remove locks and make all methods synchronized
kiranprakash154 Mar 13, 2024
d5f7572
spotless
kiranprakash154 Mar 13, 2024
41ba5be
updateCleanupKeyToCountMapOnCacheEviction
kiranprakash154 Mar 14, 2024
82a6b52
Testing
kiranprakash154 Mar 14, 2024
f2ba23f
add Reschedule back to indices service
kiranprakash154 Mar 15, 2024
adff9e1
rename updateStaleKeysCount to incrementStaleKeysCount
kiranprakash154 Mar 15, 2024
e545ba7
rename getStaleKeysCountForTesting to getStaleKeysCount
kiranprakash154 Mar 15, 2024
3dce4bd
rename threshold to stalenessThreshold
kiranprakash154 Mar 15, 2024
d05bd10
check for cleanupKey.entity == null
kiranprakash154 Mar 15, 2024
50b6240
use computeIfPresent with keycountmap
kiranprakash154 Mar 15, 2024
ab01942
log both staleKeysInCache & Staleness in debug logs
kiranprakash154 Mar 15, 2024
fdb921b
Use HashMap instead of ConcurrentMap
kiranprakash154 Mar 15, 2024
cb2cb3c
Address b/w compatibility
kiranprakash154 Mar 15, 2024
a4fb34c
remove synchronized for updateCleanupKeyToCountMapOnCacheEviction
kiranprakash154 Mar 15, 2024
15f8fc1
make cleanCache synchronized
kiranprakash154 Mar 15, 2024
1a67f00
remove shouldRemoveKey
kiranprakash154 Mar 15, 2024
dbbdff0
spotlessApply
kiranprakash154 Mar 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- [Metrics Framework] Adds support for asynchronous gauge metric type. ([#12642](https://github.com/opensearch-project/OpenSearch/issues/12642))
- Make search query counters dynamic to support all query types ([#12601](https://github.com/opensearch-project/OpenSearch/pull/12601))
- [Tiered caching] Add policies controlling which values can enter pluggable caches [EXPERIMENTAL] ([#12542](https://github.com/opensearch-project/OpenSearch/pull/12542))
- [Tiered caching] Add Stale keys Management and CacheCleaner to IndicesRequestCache ([#12625](https://github.com/opensearch-project/OpenSearch/pull/12625))

### Dependencies
- Bump `peter-evans/find-comment` from 2 to 3 ([#12288](https://github.com/opensearch-project/OpenSearch/pull/12288))
Expand Down Expand Up @@ -159,7 +160,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Fix `terms` query on `float` field when `doc_values` are turned off by reverting back to `FloatPoint` from `FloatField` ([#12499](https://github.com/opensearch-project/OpenSearch/pull/12499))
- Fix get task API does not refresh resource stats ([#11531](https://github.com/opensearch-project/OpenSearch/pull/11531))
- onShardResult and onShardFailure are executed on one shard causes opensearch jvm crashed ([#12158](https://github.com/opensearch-project/OpenSearch/pull/12158))
- Avoid overflow when sorting missing last on `epoch_millis` datetime field ([#12676](https://github.com/opensearch-project/OpenSearch/pull/12676))

### Security

Expand Down
Loading
Loading