Refactor file store #1684
Open
Refactor file store #1684
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`Path.exists` is not safe against time-of-creation, time-of-use (TOCTOU) bugs. `OpenOptions.create_new` on the other hand is atomic, so not prone to this kind of problems.
`StoreError` included the `IterError` variants, making it replaceable.
The changes in this commit were motivated due to a bug in the `StoreFile` which caused old data to be lost if the file was `open` instead of created and new data was appended. The bugfix later motivated a general name cleanup in StoreFile's methods and some minor changes in their signatures. The new StoreFile methods are: - create: create file in write only mode or fail if file exists. - load: open existing file, check integrity of content and retrieve Store. - append: add new changesets to Store. Do nothing if changeset is empty. - dump: aggregate and retrieve all stored changesets in store. - load_or_create: load if file exists, create if not, and retrieve Store.
8 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The
Store::openmethod doesn't recovers the previousStorestate saved in the file and emplaces the file pointer just after the magic bytes prefix, this later is agravated byStore::append_changesetwhich sets the file pointer after the last written changeset. The combination of both causes the lost of any previous changeset there may have been in the file.Is natural to think this shouldn't be the expected behavior, as @KnowWhoami pointed out in #1517, and the
Storeshould recover the previous changesets stored in the file store.The approach proposed in #1632 triggered a discussion about more changes in
Store, which motivated the current refactor.Besides the fix for #1517, the following methods have been changed/renamed/repurposed in Store:
create: create file and retrieve store, if exists fail.load: load changesets from file, aggregate them and return aggregated changeset andStore. If there are problems with decoding, fail.dump: aggregate all changesets and return them.load_or_create: try load or fallback to create.Fixes #1517.
Overrides #1632.
Notes to the reviewers
Load return type is
Result<(Option<C>, Store), StoreErrorWithDump>to allow methods which doesn't useWalletPersisterto get the aggregated changeset right away.Dump is kept to allow
WalletPersister::initializemethod to retrieve the aggregated changeset without forcing the inclusion of the additional parameters needed bystore::loadto the trait, which would also affect therusqliteimplementation.Changelog notice
Added:
StoreErrorenum, which includesIo,BincodeandInvalidMagicBytes.READMEforfile store.Changed:
Store::create_newtoStore::create, with new return type:Result<Self, StoreError>Store::opentoStore::load, with new return type:Result<(Option<C>, Self), StoreErrorWithDump<C>>Store::open_or_createtoStore::load_or_create, with new return type:Result<(Option<C>, Self), StoreErrorWithDump<C>>Store::aggregate_changesetstoStore::dump, with new return type:Result<Option<C>, StoreErrorWithDump<C>>FileErrortoStoreErrorAggregateChangesetsErrortoStoreErrorWithDump, which now can include all the variants ofStoreErrorin the error field.Deleted:
IterErrordeleted.Checklists
All Submissions:
I've signed all my commitsI followed the contribution guidelinesI rancargo fmtandcargo clippybefore committingNew Features:
I've added tests for the new featureI've added docs for the new featureBugfixes:
This pull request breaks the existing APII've added tests to reproduce the issue which are now passingI'm linking the issue being fixed by this PR