zbox is a command line interface (CLI) tool to understand the capabilities of 0Chain dStorage and prototype your app. The utility is built using 0Chain's goSDK library written in Go. Check out a video on how to use the CLI to create an allocation (storage volume) and upload, download, update, delete, and share files and folders to dStor on the 0Chain dStorage platform.
zbox supports the following features
- Register a Wallet
- Create an allocation
- Update an allocation
- Upload a file to dStorage
- Download the uploaded file from dStorage
- Update the uploaded file on dStorage
- Delete the uploaded file on dStorage
- Share the uploaded file on dStorage to the public
- List the uploaded files and folders
- Copy uploaded files to another folder path on dStorage
- Upload encrypted files to dStorage
- Share an encrypted file using proxy re-encryption (PRE) with your friend
- List allocations
- Sync your local folder to remote
- Get wallet information
- Get Allocation info
- Get meta data of files
- Rename an object in allocation
- Get file stats
- Repair a file on dStorage
- Cancel allocation
- Finalize allocation
- Challenge pool information
- List blobbers
- Read pool
- Storage SC configurations
- Stake pool
- Write pool
- Costs
- Blobber
- Move uploaded files to another folder path on dStorage
- Add Collaborator for a file
- Remove Collaborator for a file
zbox CLI provides a self-explaining "help" option that lists commands and parameters they need to perform the intended action
git clone https://github.com/0chain/zboxcli.git
Go V1.12 or higher.
Before you start playing with zbox, you need to access the blockchain. Go to network folder in the repo, and choose a network. Copy it to your ~/.zcn folder and then rename it as config.yaml file.
mkdir ~/.zcn
cp network/one.yaml ~/.zcn/config.yaml
Sample config.yaml
---
block_worker: http://localhost:9091
signature_scheme: bls0chain
min_submit: 50 # in percentage
min_confirmation: 50 # in percentage
confirmation_chain_length: 3
We use blockWorker to connect to the network instead of giving network details directly, It will fetch the network details automatically from the blockWorker's network API.
By default it will use the miner/sharder values which it will get using the block_worker_url/network
. In case you want to override those values and give custom miner/sharder to use, You can create a network.yaml
in your ~/.zcn (config) folder and paste the miner/sharder values in below format.
miners:
- http://localhost:7071
- http://localhost:7072
- http://localhost:7073
sharders:
- http://localhost:7171
Note: This is also useful for the Mac OS users running local cluster and having trouble with docker internal IPs (block_worker return docker IPs in local)
The zbox command line uses the ~/.zcn/config.yaml file at runtime to point to the network specified in that file.
Note in this document, we will show only the commands, response will vary depending on your usage, so may not be provided in all places.
When you run zbox with no arguments, it will list all the supported commands.
Command
./zbox
Response
0Box is a decentralized storage application written on the 0Chain platform.
Complete documentation is available at https://0chain.net
Usage:
zbox [command]
Available Commands:
alloc-cancel Cancel an allocation
alloc-fini Finalize an expired allocation
bl-info Get blobber info
bl-update Update blobber settings by its delegate_wallet owner
copy copy an object(file/folder) to another folder on blobbers
cp-info Challenge pool information.
delete delete file from blobbers
download download file from blobbers
get Gets the allocation info
get-download-cost Get downloading cost
get-upload-cost Get uploading cost
getwallet Get wallet information
help Help about any command
list list files from blobbers
listallocations List allocations for the client
ls-blobbers Show active blobbers in storage SC.
meta get meta data of files from blobbers
move move an object(file/folder) to another folder on blobbers
newallocation Creates a new allocation
register Registers the wallet with the blockchain
rename rename an object(file/folder) on blobbers
rp-create Create read pool if missing
rp-info Read pool information.
rp-lock Lock some tokens in read pool.
rp-unlock Unlock some expired tokens in a read pool.
sc-config Show storage SC configuration.
share share files from blobbers
sp-info Stake pool information.
sp-lock Lock tokens lacking in stake pool.
sp-pay-interests Pay interests not payed yet.
sp-unlock Unlock tokens in stake pool.
sp-user-info Stake pool information for a user.
start-repair start repair file to blobbers
stats stats for file from blobbers
sync Sync files to/from blobbers
update update file to blobbers
updateallocation Updates allocation's expiry and size
upload upload file to blobbers
version Prints version information
wp-info Write pool information.
wp-lock Lock some tokens in write pool.
wp-unlock Unlock some expired tokens in a write pool.
update-attributes Update file attributes.
Flags:
--config string config file (default is config.yaml)
--configDir string configuration directory (default is $HOME/.zcn)
-h, --help help for zbox
--verbose prints sdk log in stderr (default false)
--wallet string wallet file (default is wallet.json)
Use "zbox [command] --help" for more information about a command.
Command register registers a wallet that will be used both by the blockchain and blobbers, and is created in the ~/.zcn directory. If you have created a wallet with another network, you will need to remove and recreate it. If you want to create multiple wallets with multiple allocations, make sure you store the wallet information. zbox uses the keys in ~/.zcn/wallet.json when it executes the commands.
Command
./zbox register
Response
ZCN wallet created
Wallet registered
Command newallocation reserves hard disk space on the blobbers. Let's see the parameters it takes by using --help
Command
./zbox newallocation --help
Response
Creates a new allocation
Usage:
zbox newallocation [flags]
Flags:
--allocationFileName string --allocationFileName allocation.txt (default "allocation.txt")
--data int --data 2 (default 2)
--expire duration duration to allocation expiration (default 720h)
-h, --help help for newallocation
--lock string lock write pool with given number of tokens, or use 'auto' (default "0")
--mcct duration max challenge completion time, optional, default 1 hour
--parity int --parity 2 (default 2)
--read_price string select blobbers by provided read price range, use form 0.5-1.5, default is [0; inf)
--size int --size 10000 (default 2147483648)
--write_price string select blobbers by provided write price range, use form 1.5-2.5, default is [0; inf)
Global Flags:
--config string config file (default is config.yaml)
--configDir string configuration directory (default is $HOME/.zcn)
--verbose prints sdk log in stderr (default false)
--wallet string wallet file (default is wallet.json)
As you can see the newallocation command takes allocationFileName where the volume information is stored locally. All the parameters have default values. With more data shards, you can upload or download files faster. With more parity shards, you have higher availability.
Create a new allocation with default values. If you have not registered a wallet,
it will automatically create a wallet. The newallocation also, on success,
creates related write pool. Use --lock
flag to add some tokens to the pool
in the transaction.
Command
./zbox newallocation
Response
Allocation created : d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac
Also, the allocation information is stored under $Home/.zcn/allocation.txt
Command updateallocation updates hard disk space and expiry on the blobbers. Let's see the parameters it takes by using --help
Command
./zbox updateallocation --help
Response
Updates allocation's expiry and size
Usage:
zbox updateallocation [flags]
Flags:
--allocation_id string allocation ID
--expiry duration adjust storage expiration time, duration (default 720h)
-h, --help help for updateallocation
--size int adjust allocation size, bytes (default 2147483648)
Global Flags:
--config string config file (default is config.yaml)
--configDir string configuration directory (default is $HOME/.zcn)
--verbose prints sdk log in stderr (default false)
--wallet string wallet file (default is wallet.json)
Create a new allocation with default values. If you have not registered a wallet, it will automatically create a wallet.
Command
./zbox updateallocation --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac
Response
Allocation updated with txId : fb84185dae620bbba8386286726f1efcd20d2516bcf1a448215434d87be3b30d
You can see more txn details usin above txID in block explorer.
Finalize an expired allocation. It makes sure all blobbers has got min_lock_demand, unlocks all related pools. Moves tokens from write pool to allocation owner.
./zbox finialloc --allocaiton <allocaiton_id>
Use upload command to upload a file. By using help for this command, you will see it takes parameters:
- --allocation -- the allocation id from the newallocation command
- --localpath -- absolute path to the file on your local system
- --remote path -- remote path where you want to store. It should start with "/"
- --thumbnailpath -- Local thumbnail path of file to upload
- --encrypt -- [OPTIONAL] pass this option to encrypt and upload the file
- --commit -- [OPTIONAL] pass this option to commit the metadata transaction
- --attr-who-pays-for-reads [OPTIONAL] set payer for downloads; default is 3rd_party, it can be 'owner' or '3rd_party'
Command
./zbox upload --localpath <absolute path to file>/hello.txt --remotepath /myfiles/hello.txt --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac
Response
Status completed callback. Type = application/octet-stream. Name = hello.txt
Use upload command with optional encrypt parameter to upload a file in encrypted format. This can be downloaded as normal from same wallet/allocation or utilize Proxy Re-Encryption facility (see download command).
Command
./zbox upload --encrypt --localpath <absolute path to file>/sensitivedata.txt --remotepath /myfiles/sensitivedata.txt --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac
Response
Status completed callback. Type =
Use download command to download your own or a shared file.
- --allocation string Allocation ID
- --commit Pass this option to commit the metadata transaction
- --authticket string Auth ticket fot the file to download if you dont own it
- --localpath string Local path of file to download
- --lookuphash string [OPTIONAL] The remote lookuphash of the object retrieved from the list
- --remotepath string Remote path to download
- -t, --thumbnail pass this option to download only the thumbnail
- -e, --endblock int pass this option to download till specific block number
- -s, --startblock int pass this option to download from specific block number
- --rx_pay used to download by authticket; pass true to pay for download yourself
Command
./zbox download --authticket eyJjbGllbnRfaWQiOiIiLCJvd25lcl9pZCI6IjRiZjI4ODU5NzgzMjNiMmU0OGUyNGM0ZTNkODkwYTA1MzQwM2E3MDk3NDE3MDljMzA1YjAxZjE5ZDk2NDFhYTgiLCJhbGxvY2F0aW9uX2lkIjoiM2MwZDMyNTYwZWExOGQ5ZDBkNzY4MDgyMTZhOWM2MzRmNjYxOTc5ZDI5YmE1OWNjOGRhZmNjYjNlNWI5NTM0MSIsImZpbGVfcGF0aF9oYXNoIjoiNDE4NjVmMGM2YWFhNTcxM2VkMzkxZWJkZjgyMjU1MmZjNmNmYjU5YTg3YTI2MTY4MjgyNDJiYTNjYTBkY2U0OSIsImZpbGVfbmFtZSI6ImhvcnNlLmpwZyIsInJlZmVyZW5jZV90eXBlIjoiZiIsImV4cGlyYXRpb24iOjE1Njg3NTQ0ODQsInRpbWVzdGFtcCI6MTU2MDk3ODQ4NCwic2lnbmF0dXJlIjoiYjhkZWNhNzM4YjgyNGRiNmNlNzc0NDY1N2FlZmNiNzUzZTYxOWQ4MmJhODEzMjIzYWQ3MGI2NTlkOTQxNDM2YTVkMzQ0N2E5ZmUwNzE1NGYwMThmYjk5NDkyNDQ5ZDk5NmNjMmQ5M2RkMWM0NTJkYzgzNDEyYjVhZTNkMmFmMDEifQ== --allocation 3c0d32560ea18d9d0d76808216a9c634f661979d29ba59cc8dafccb3e5b95341 --remotepath /myfiles/horse.jpeg --lookuphash 41865f0c6aaa5713ed391ebdf822552fc6cfb59a87a2616828242ba3ca0dce49 --localpath ../horse.jpeg
Note: You can download by using only 1 on the below combination
- --remotepath , --allocation
- --authticket
Response
Downloaded file will be in the localpath specified.
Use update command to update content of an existing file in the remote path. Similar to upload command.
Use delete command to delete your file on allocation. --allocation string Allocation ID --commit Pass this option to commit the metadata transaction --remotepath string Remote path to download Command
./zbox delete --allocation 3c0d32560ea18d9d0d76808216a9c634f661979d29ba59cc8dafccb3e5b95341 --remotepath /myfiles/horse.jpeg
Response
File successfully deleted (Can be verified using list)
Use share command to generate an authtoken that provides authorization to the holder to the specified file on the remotepath. --allocation string Allocation ID --clientid string ClientID of the user to share with. Leave blank for public share --encryptionpublickey string Encryption public key of the client you want to share with (from getwallet command ) --remotepath string Remote path to share
Command
./zbox share --allocation 3c0d32560ea18d9d0d76808216a9c634f661979d29ba59cc8dafccb3e5b95341 --remotepath /myfiles/hello.txt
Use clientid and encryptionpublickey of the user to share with.
Command
./zbox share --allocation 3c0d32560ea18d9d0d76808216a9c634f661979d29ba59cc8dafccb3e5b95341 --remotepath /myfiles/hello.txt --clientid d52d82133177ec18505145e784bc87a0fb811d7ac82aa84ae6b013f96b93cfaa --encryptionpublickey +BV37Ip05OdHo+Sz4N8xgCrACCUOVLnQICU1IiJq8uU=
Response
Response contains auth token an encrypted string that can be shared.
Use list command to list files in given path. By using help for this command, you will see it takes parameters:
- --allocation string --Allocation ID
- --remotepath string --Remote path to list from (Required for --allocation)
- --authticket string --Auth ticket fot the file to download if you dont own it
- --lookuphash string --The remote lookuphash of the object retrieved from the list
Command
./zbox list --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac --remotepath /myfiles
Response
Response will be a list with information for each file/folder in the given path. The information includes lookuphash which is require for download via authticket (Optional file list in json format)
Use copy command to copy file to another folder path in dStorage. By using help for this command, you will see it takes parameters:
- --allocation string --Allocation ID
- --destpath string Destination path for the object. Existing directory the object should be copied to.
- --remotepath string Remote path of object to copy
Command
./zbox copy --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac --remotepath /file.txt --destpath /existingFolder/file.txt
Use move command to move file to another folder path in dStorage. By using help for this command, you will see it takes parameters:
- --allocation string --Allocation ID
- --destpath string Destination path for the object. Existing directory the object should be copied to.
- --remotepath string Remote path of object to move
Command
./zbox move --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac --remotepath /file.txt --destpath /existingFolder/file.txt
Use list allocations command to list all allocations for a client.
Command
./zbox listallocations
Response
Response will be table format output in which all of your allocations will be listed with their ID, SIZE, EXPIRATION, DATASHARDS and PARITYSHARDS
Sync command helps to sync all files in localfolder recursively to remote.
Command
./zbox sync --help
Response
Sync all files to/from blobbers from/to a localpath
Usage:
zbox sync [flags]
Flags:
--allocation string Allocation ID
--commit pass this option to commit the metadata transaction - only works with uploadonly
--excludepath stringArray Remote folder paths exclude to sync
-h, --help help for sync
--localcache string Local cache of remote snapshot.
If file exists, this will be used for comparison with remote.
After sync complete, remote snapshot will be updated to the same file for next use.
--localpath string Local dir path to sync
--uploadonly pass this option to only upload/update the files
Global Flags:
--config string config file (default is config.yaml)
--configDir string configuration directory (default is $HOME/.zcn)
--verbose prints sdk log in stderr (default false)
--wallet string wallet file (default is wallet.json)
Command
./zbox sync --localpath /myLocalFolder --remotepath / --allocation 06e227bc283cb9ebc98c164f8f3fd4f02e70f76ec93cb8b41b276146db3f329f --localcache /someFolder/Localcache.txt
Response
It will sync your localpath with the remote and do all the requried CRUD operations.
Use command get to get additional wallet information including Encryption Public Key required for Proxy Re-Encryption.
Command
./zbox getwallet
Response
Response will give details for current selected wallet (or wallet file specified by optional --wallet parameter)
Use command get to get the information about the allocation like total size of the allocation, used size, number of challenges and the result of that, etc.
Command
./zbox get --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac
Response
Response will have information about blobbers allocated and stats for the allocation. Stats contain important information about the size of the allocation, size used, number of write markers, and challenges passed/failed/open/redeemed
Use command meta to get meta data for the given remote file. Use with help to know more about possible flags.
- --allocation string -- Allocation ID
- --authticket string -- Auth ticket fot the file to download if you dont own it
- --lookuphash string -- The remote lookuphash of the object retrieved from the list
- --remotepath string -- Remote path to list from
Command
./zbox meta --allocation 0834e2848e3006714423f485cfc1c22266368d655026152bd762cd0cdd0a4aeb --remotepath /file.txt
With authticket
./zbox meta --authticket <authticket in encoded hash string> --lookuphash <optional>
Response
Response will be meta data for the given filepath/lookuphash (if using authTicket)
rename command helps in renaming a file already exists in dStorage.
- --allocation string -- Allocation ID
- --destname string -- New Name for the object (Only the name and not the path). Include the file extensio*if applicable
- --remotepath string -- Remote path of object to rename
Command
./zbox rename -- allocation 0834e2848e3006714423f485cfc1c22266368d655026152bd762cd0cdd0a4aeb --remotepath /file1.txt --destname file2.txt
stats command helps in getting upload, download and challenge information on a file.
Command
./zbox stats --allocation 3c0d32560ea18d9d0d76808216a9c634f661979d29ba59cc8dafccb3e5b95341 --remotepath /myfiles/horse.jpg
Use repair command to repair a file. By using help for this command, you will see it takes parameters:
- --allocation -- the allocation id from the newallocation command
- --localpath -- absolute path to the file on your local system
- --remote path -- remote path where you want to store. It should start with "/"
Command
./zbox repair --localpath <absolute path to file>/hello.txt --remotepath /myfiles/hello.txt --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac
Response
Status completed callback. Type = application/octet-stream. Name = hello.txt
Use command add-collab to add a collaborator for a file
- --allocation string -- Allocation ID
- --collabid string -- Collaborator's clientID
- --remotepath string -- Remote path to list from
Command
./zbox add-collab --allocation 0834e2848e3006714423f485cfc1c22266368d655026152bd762cd0cdd0a4aeb --remotepath /file.txt
--collabid 271aeadc4c5f909ad967fa62fbeb261aaca48aec90ddbf9b48f269cebde623c7
Response
Response will be a confirmation that collaborator is added on all blobbers for the given file You can check all collaborators for a file in metadata json response
Use command delete-collab to remove a collaborator for a file
- --allocation string -- Allocation ID
- --collabid string -- Collaborator's clientID
- --remotepath string -- Remote path to list from
Command
./zbox delete-collab --allocation 0834e2848e3006714423f485cfc1c22266368d655026152bd762cd0cdd0a4aeb --remotepath /file.txt
--collabid 271aeadc4c5f909ad967fa62fbeb261aaca48aec90ddbf9b48f269cebde623c7
Response
Response will be a confirmation that collaborator is removed on all blobbers for the given file
Cancel allocation immediately and return all tokens from challenge pool back to user (to write pool). This case blobber will not give their min lock demand. If blobbers already got some tokens, the tokens will not be returned.
It returns tokens to user's write pool, creating one expired if necessary.
./zbox alloc-cancel --allocation ALLOCATION_ID
Finalize an expired allocation. When allocation expired, after its challenge completion time (after the expiration) an allocation can be finalized by owner or one of allocation blobbers.
./zbox alloc-fini --allocation ALLOCATION_ID
Challenge pool brief information.
./zbox cp-info --allocation ALLOCATION_ID
Example
BALANCE | START | EXPIRE | FINIALIZED
+--------------+-------------------------------+-------------------------------+------------+
0.0039020457 | 2020-04-24 03:54:35 +0400 +04 | 2020-04-26 03:56:35 +0400 +04 | false
Where time bounds and the finalization are allocations related. Balance is current challenge pool balance.
Show active blobbers in storage SC.
./zbox ls-blobbers
Create read pool if missing.
./zbox rp-create
Read pool information.
./zbox rp-info
Filtering result for an allocation.
./zbox rp-info --allocation ALLOCATION_ID
Lock some tokens in read pool associated with an allocation. The tokens will be divided between allocation blobbers by their read price.
./zbox rp-lock --allocation ALLOCATION_ID --duration 40m --tokens 1
Unlock an expired read pool by pool id. See rp-info
for the POOL_ID and
the expiration.
./zbox rp-unlock --pool_id POOL_ID
Show storage SC configuration.
./zbox sc-config
Stake pool related commands.
Stake pool information and settings.
./zbox sp-info --blobber_id BLOBBER_ID
Lock crates delegate pool for current client and given blobber. The tokens locked for the blobber stake and can be unlocked any time, excluding where the tokens held by opened offers. The tokens collect interests.
./zbox sp-lock --blobber_id BLOBBER_ID --tokens 1.0
Unlock a stake pool by pool owner.
./zbox sp-unlock --blobber_id BLOBBER_ID --pool_id POOL_ID
Get information about all stake pools of current user.
./zbox sp-user-info
Some changes in stake pool pays all pending rewards to calculate next rewards
correctly and don't complicate stake pool. But if there is no changes interests
will not be payed. The sp-pay-interests
can be used to pay the interests. The
command never fails. It pays interests for all delegates. Use sp-info
to check
interests can be payed.
./zbox sp-pay-interests --blobber_id BLOBBER_ID
Write pool information.
For all write pools.
./zbox wp-info
Filtering by allocation.
./zbox wp-info --allocation ALLOCATION_ID
Lock tokens in a write pool associated with an allocation. All tokens will be divided between allocation blobbers depending on their write price.
./zbox wp-lock --allocation ALLOCATION_ID --duration 40m --tokens 1
Unlock an expired write pool by its POOL_ID. See wp-info for the id and the expiration. An expired write pool, associated with an allocation, can be locked until allocation finalization even if it's expired. It possible in cases where related blobber doesn't give their min lock demands. The finalization will pay the demand an unlock the pool.
./zbox wp-unlock --pool_id POOL_ID
How much a remote file downloading.
./zbox get-download-cost --allocation ALLOC_ID --remotepath /path/file.ext
Also, there are authticket
and lookuphash
flags to get the cost for
non allocation owners.
How much a local file uploading.
./zbox get-upload-cost --allocation ALLOC_ID --localpath ./path/file.ext
Get blobber detailed information.
./zbox bl-info --blobber_id BLOBBER_ID
Update blobber settings.
./zbox bl-update --blobber_id BLOBBER_ID [settings to update]
Use ./zbox bl-update --help
to get list of setting can be updated.
Update file attributes.
./zbox update-attributes --allocation ALLOC_ID --remotepath=/dir/file.txt \
--who-pays-for-reads=3rd_party
Only one attribute is currently supported: who-pays-for-reads that can be:
owner
, where allocation owner pays for own and 3rd_party reads3rd_party
, where 3rd party readers pays for their downloads themselves
Since zboxcli is wrapper to gosdk methods, streaming feature should be implemented together with player for each platforms (Android, IOS, Mac, Windows).
IOS documentation: https://github.com/0chain/0box-ios
Android documentation: https://github.com/0chain/0boxAndroid
Mac documentation: https://github.com/0chain/0BoxSyncMac
For platforms using zboxcli commands, implementation are similar to platforms above, i.e.:
- Download file with downloadFileByBlocks method
- Read chunked files to byte array (inputstream)
- Add byte array to custom media source of player
Improvements was done in
downloadFileByBlocks - properly returns file-chunks with correct byte range, gosdk v1.2.4 and above only.
getFileMeta - returns actuaBlockNumbers and actualFileSize (exclude thumbnail size)
getFileMetaByAuth - same updates as getFileMeta
listAllocation - returns actuaBlockNumbers and actualFileSize (exclude thumbnail size)
- Both
rp-info
andrp-lock
are not working.
./zbox rp-info
Failed to get read pool info: error requesting read pool info: consensus_failed: consensus failed on sharders
This can happen if read pool is not yet created for wallet. Read pool is usually created when new wallet is created by zbox
or zwallet
.
However, if wallet is created through zwallet recoverwallet
, read pool may not have been created. Simply run zbox rp-create