TwoCommit

Problem

There can occur situations, when during connection commit blobbers will have different states. Less than enough to recover data blobbers can receive write marker and commit connection S1, the rest of bobbers can stay with previous uncommitted state S0 (broken). If a client will decide to repair such allocation, there won't be enough bobbers to restore committed state and there is no way to rollback, since blobbers have already merged changes.

It can happen, when blobbers will crash during connection commit, or client will send commit only partially to blobbers.

Solution

To provide a way to get back to previous correct state (S0), even if it is committed on part of the network, we introduce double commit approach. We add new state of the filesystem "pre-committed" S1 (in yellow)

Implementation

We should assume that client-blobber interactions (data upload, connection_commit) are decoupled from blobber-blockchain-validator interactions (write_marker submit, challenge complete). It is achieved mainly by using chains of writemarkers during completing challenges, which in order allows blobbers not to store all the versions of allocations, but use the latest one and do not store any earlier versions.

Simple flow

Pre-committed section

At the moment changes are stored to getAllocTempDir (tmp) directory. We will add new directory getAllocPreCommitDir (precommit). After changes are pre-committed they are moved to this directory from getAllocTempDir. It will allow us to store S1 (pic) in the different isolated sandbox.

Pre-commit event

It is the main part of double-commit protocol, we pre-commit on CommitWrite, but instead of connectionObj.ApplyChanges we move changes to getAllocPreCommitDir dir.

CommitWrite

When client commits write blobber does several actions:

• commit previously pre-committed data with connectionObj.ApplyChanges and moving them from getAllocPreCommitDir to filesystem as before on commit
• pre-commit current data that is speculative, move it to getAllocPreCommitDir from getAllocTempDir

##Rollback We add Rollback, to be able to rollback pre-committed state S1(pic) to committed S0(pic) state. We do not delete pre-committed changes though, since these changes could be already committed to blockchain and challenged, instead we store them locally for some time, but remove from getAllocPreCommitDir to some other temp dir. These changes won't be used anyhow in the future, only to complete challenges.

Data manipulation

/v1/file and /v1/dir uses pre-committed state to manipulate it, so instead of merged state before, we will use files from connectionObj.ApplyChanges section to build state upon

Empty commit

It is suggested to send empty writemarker after client session is ended to commit all the changes and empty pre-commit and speculative sections.

Challenge protocol

Challenge protocol do not change, everything works the same, the only difference is that rolled back data can be challenged and processed the same way as regular challenge. It is added for generality, to change as less as possible.