TwoCommit

Problem

There can occur situations when during connection commit blobbers will have different states, less than enough to recover data can receive write marker and commit connection, another part of bobbers can stay with previous uncommitted state (broken). If a client will decide to repair such allocation, there won't be enough bobbers to restore committed state and there is no way to rollback since blobbers already merged changes.

It can happen when blobbers will crash during connection commit, or client will send commit only partially to blobbers.

Solution

To provide a way to get back to previous correct state (S0) even if it is committed on part of the network we introduce double commit approach. We add new state of the filesystem pre-committed (in yellow)

Implementation

We should assume that client-blobber interactions (data upload, connection_commit) are decoupled from blobber-blockchain-validator interactions (write_marker submit, challenge complete). It is achieved mainly by using chains of writemarker during completing challenges, which allows blobbers not to store all the versions of allocations, but use the latest one and do not store any earlier versions.

Pre-committed section

At the moment changes are stored to getAllocTempDir (tmp) directory. We will add new directory getAllocPreCommitDir (precommit). After changes are pre-committed they are moved to this directory from getAllocTempDir. It will allow us to store S1 (pic) in the different isolated sandbox.

Pre-committed action

It is the main part of double-commit protocol, we precommit on CommitWrite, but instead of connectionObj.ApplyChanges we move changes to getAllocPreCommitDir dir.

CommitWrite

When client commits write we do several actions:

• commit previously pre-committed data with connectionObj.ApplyChanges for them and moving them from getAllocPreCommitDir to filesystem as before on commit
• pre-commit current data that is committed, move it to getAllocPreCommitDir from getAllocTempDir

##Rollback We add Rollback, to be able to rollback pre-committed state S1(pic) to committed S0(pic) state. We do not delete pre-committed changes, since they could be already committed to blockchain and challenged, instead we store them locally for some time, but remove from getAllocPreCommitDir to some other temp dir. These changes won't be used anyhow in the future, only to complete challenges.

Data manipulation

/v1/file and /v1/dir uses pre-committed state to manipulate it, so instead of merged state before, we will use files from connectionObj.ApplyChanges section to build state upon

##Challenge protocol Challenge protocol do not change, everything works the same, the only difference is that rolled back data can be challenged and processed the same way as regular challenge. It is added for generality, to change as less as possible.