Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CBG-3203: Don't store channel history for non-leaf revisions #6368

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

bbrks
Copy link
Member

@bbrks bbrks commented Aug 15, 2023

CBG-3203

  • Moves channel history storage for non-winning leaf-revisions into a channelMap during marshal/unmarshal.
  • Use _sync.Channels to build set of activeChannels returned by IsChannelRemoval

Integration Tests

Copy link
Collaborator

@adamcfraser adamcfraser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look fine, but the calls to .Leaves() in marshal and unmarshal seem like the have the potential for non-trivial performance head when marshalling/unmarshalling metadata. Posted some questions about whether it's necessary.

db/revtree.go Outdated Show resolved Hide resolved
db/revtree.go Show resolved Hide resolved
db/revtree.go Outdated Show resolved Hide resolved
db/revtree.go Show resolved Hide resolved
db/revtree.go Outdated Show resolved Hide resolved
db/crud.go Outdated Show resolved Hide resolved
db/revision_cache_interface.go Outdated Show resolved Hide resolved
@bbrks bbrks force-pushed the CBG-3203 branch 2 times, most recently from 97e6e10 to ce69d5a Compare November 2, 2023 13:42
@bbrks bbrks assigned adamcfraser and unassigned bbrks Nov 3, 2023
@bbrks
Copy link
Member Author

bbrks commented Nov 3, 2023

@adamcfraser I think this PR is finally good for a fresh review! Thanks

Copy link
Collaborator

@adamcfraser adamcfraser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the approach, especially how it minimizes the changes. Couple of questions on the details.

db/revtree.go Outdated Show resolved Hide resolved
db/crud.go Show resolved Hide resolved
@bbrks bbrks assigned bbrks and adamcfraser and unassigned adamcfraser and bbrks Nov 20, 2023
@bbrks bbrks requested a review from adamcfraser November 21, 2023 17:07
// As a special case, you don't need channel access to see a deletion revision,
// otherwise the client's replicator can't process the deletion (since deletions
// usually aren't on any channels at all!) But don't show the full body. (See #59)
// Update: this applies to non-deletions too, since the client may have lost access to
// the channel and gotten a "removed" entry in the _changes feed. It then needs to
// incorporate that tombstone and for that it needs to see the _revisions property.
if revid == "" || doc.History[revid] == nil {
if revid == "" || doc.History[revid] == nil || errors.Is(err, ErrMissing) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the comment on line 700, it seems like we want to bypass the security check for tombstones or removals (so that we successfully set bodyBytes to either EmptyDocument or RemovedRedactedDocument below). I'm wondering if the ErrMissing handling is going to break this, in particular for removals. I went looking for a test that covers this path - TestGetRemovedAsUser seems close but I don't think it purges the removal from the rev cache before attempting to retrieve. I think it would be good to extend TestGetRemovedAsUser for this scenario to see whether it changes before/after this PR:

  • removal revision is non-leaf
  • removal revision is not resident in the rev cache
  • verify removal is still returned

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same might be true for tombstones, although non-leaf tombstones are more of a corner case. It would be common for removals to be non-leaf revisions.

Copy link
Member Author

@bbrks bbrks Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My interpretation of the comment was that an old rev is neither removed from a channel, nor a tombstone. Is that incorrect?

An old revision we don't have channel info for any more seems the same as somebody requesting a pruned revision, or doc1 / 123-invalid which would return ErrMissing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of the usage of IsChannelRemoval in revCacheLoaderForDocument , where we check the channel history to identify removals for revisions that aren't in the rev tree, and return a removal instead of 404 for those. I wasn't clear about the interaction (if any) between that handling and the changes here.

Copy link
Member Author

@bbrks bbrks Nov 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, will follow up and get test coverage for a removal that isn't cached and ensure behaviour doesn't change with this PR.

bbrks added 26 commits April 8, 2024 15:15
…ct and isn't storing non-leaf channelmap entries)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants