Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental change to not care about cversion for Watched collections with persistent recursive watch #120

Open
wants to merge 1 commit into
base: fs/branch_9_2
Choose a base branch
from

Conversation

patsonluk
Copy link
Collaborator

Description

An experimental branch/PR to explore using persistent recursive watch without maintain the versioning of PRS entries (whether it's cversion or pzxid as proposed in #105 )

There are various challenges in #105, and one of the biggest issues was to keep the versioning of PRS entries. So this PR explores whether it's possible to ignore such versioning for "watched" collections. As watched collections are supposed be updated by watched events/notification (and that ONLY after the initial fetch), therefore versioning is not important to watched collections

This branch passed both test cases in PerReplicaStatesIntegrationTest and TestPerReplicaStates. Ran PerReplicaStatesIntegrationTest#testZkNodeVersions (exclude the non PRS collection) and captured metrics similar to https://fullstory.atlassian.net/browse/SAI-4544?focusedCommentId=259534.

It's found that this PR improves (ie reduce fetches) for collection creation, add/remove replica. but for node restart, it's pretty much the same as 9.2. This is probably due to the fact that most fetches for restart are actually triggered from ZkStateReader#waitForState (

registerDocCollectionWatcher(collection, watcher);
// wait for the watcher predicate to return true, or time out
if (!latch.await(wait, unit))
throw new TimeoutException(
"Timeout waiting to see state for collection="
+ collection
+ " :"
+ docCollection.get());
return docCollection.get();
} finally {
removeDocCollectionWatcher(collection, watcher);
) which for each core, register/unregister Doc collection listener multiple times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant