bug: uploading the same files from different devices ends up paying for some chunks more than once

[happybeing]

Here's the output of safe files upload -r from a VPS after using rsync to transfer the files from the laptop to the VPS:

Splitting and uploading "/tmp/vps-wallet-tmpdir/v1-awv-type-site/" into 76 chunks
**************************************
*          Uploaded Files            *
**************************************
Uploaded "global.css" to address d24c90d3c74ca41994073775480d4e4b4a4c6e095430ed7cf84f1d5a0d4ff105
Uploaded "tetris.html" to address 4e9dee9d0b3f864bb36df2730cb7ad3089e5847d03058f04042b44ad38ba6410
Uploaded "global-thermonuclear-war.html" to address 11247c6c6dbe3627d98368a929b9a97128cd5ae5ae394cab22985aa02694a5e6
Uploaded "bundle.js.map" to address e9f09b4dd6ae5b2add52877c7a002edbd90e561415c844e8e97cf6f8ae074a5b
Uploaded "bundle.css.map" to address 32752bb27883649f422de9e91c47f50595fab161bf708833fefe9a291903079f
Uploaded "bundle.js" to address ff55fe6fbdb87aac1e331e3b9f1dfc4e1a9cf87f8ef567e780b8f051eaaba7c7
Uploaded "breakout.html" to address 4211031d3ef7024c8a5e60b47d3c76d2cf058efaa13160e1edbb2e5f587eafa0
Uploaded "ant-invasion.html" to address b59fc09324b1bb6d78847d1bc175fcab7463d8f1eb33c7966ada03e6c03ad8cc
Uploaded "index.html" to address 5466ea2f7d066e61bfd56cac1a9104d79f5e23e96163313e063140ac43548a3c
Uploaded "riddim-ant.jpeg" to address 83d14bb4df8c4963087343026692759a383b4917592c09ee5a25b3d9e51d8cf1
Uploaded "bundle.css" to address f3c12b047366359b759e69867c9402d625933e2c243b0138c7843beb217cc3ec
Uploaded "bundle.js.map" to address c37458cfe80d07ac02a9a1c6fbcf4bd14529fd24cdbd811926de7dd92a4ea4ac
Uploaded "favicon.png" to address 2393fc4b9b138ef532b2ca72aa356f7fa863bfee4c267b04835738e5d471b461
Uploaded "bundle.css" to address 364bc51cc292d66e6fb959f2c55bd2f12e6e49388a877a2dc497b7bda285f6ff
Uploaded "another-ant.png" to address fe0d0b9bb2ac1bc85bc410a59ea7c0e462e420c1540e7a860239bba68c820f4f
Uploaded "bundle.js" to address 4f9dee9055bd3179ecf5eaf357b02f482b8004d40d377cf886fbf4df3964f8d8
Uploaded "index.html" to address 0dd81e4c03883458db7ced33c63a77c5483f7495c9265716147b941f7a4cdc6a
Uploaded "henry-the-anth.html" to address 8ad73285c6e3a6c7a2e1a2f3d359cecd99b84470ee22f9d95e573b9701890d7d
Uploaded "index-ants.png" to address b1ccf5989cc287532cae5a98adabc03bfc2cd95368c09e41dd066335a5e2a8f9
Among 76 chunks, found 52 already existed in network, uploaded the leftover 24 chunks in 2 minutes 1 seconds
**************************************
*          Payment Details           *
**************************************
Made payment of NanoTokens(25) for 24 chunks
Made payment of NanoTokens(25) for royalties fees
New wallet balance: 0.000000450
Completed with Ok(()) of execute "Files(Upload { file_path: \"/tmp/vps-wallet-tmpdir/v1-awv-type-site/\", batch_size: 16, make_data_public: true, retry_strategy: Quick })"

Subsequently, uploading the original files from the laptop pays for three of the chunks again:

Splitting and uploading "/home/mrh/src/safe-browser/awe-sites/awv-type-site/v1-awv-type-site" into 25 chunks
**************************************
*          Uploaded Files            *
**************************************
Uploaded "global.css" to address d24c90d3c74ca41994073775480d4e4b4a4c6e095430ed7cf84f1d5a0d4ff105
Uploaded "ant-invasion.html" to address b59fc09324b1bb6d78847d1bc175fcab7463d8f1eb33c7966ada03e6c03ad8cc
Uploaded "bundle.css.map" to address 32752bb27883649f422de9e91c47f50595fab161bf708833fefe9a291903079f
Uploaded "global.css" to address d24c90d3c74ca41994073775480d4e4b4a4c6e095430ed7cf84f1d5a0d4ff105
Uploaded "ant-invasion.html" to address b59fc09324b1bb6d78847d1bc175fcab7463d8f1eb33c7966ada03e6c03ad8cc
Uploaded "bundle.css.map" to address 32752bb27883649f422de9e91c47f50595fab161bf708833fefe9a291903079f
Uploaded "bundle.js.map" to address c37458cfe80d07ac02a9a1c6fbcf4bd14529fd24cdbd811926de7dd92a4ea4ac
Uploaded "another-ant.png" to address fe0d0b9bb2ac1bc85bc410a59ea7c0e462e420c1540e7a860239bba68c820f4f
Uploaded "bundle.js" to address ff55fe6fbdb87aac1e331e3b9f1dfc4e1a9cf87f8ef567e780b8f051eaaba7c7
Uploaded "bundle.css" to address f3c12b047366359b759e69867c9402d625933e2c243b0138c7843beb217cc3ec
Uploaded "bundle.js.map" to address e9f09b4dd6ae5b2add52877c7a002edbd90e561415c844e8e97cf6f8ae074a5b
Uploaded "index.html" to address 0dd81e4c03883458db7ced33c63a77c5483f7495c9265716147b941f7a4cdc6a
Uploaded "breakout.html" to address 4211031d3ef7024c8a5e60b47d3c76d2cf058efaa13160e1edbb2e5f587eafa0
Uploaded "index.html" to address 5466ea2f7d066e61bfd56cac1a9104d79f5e23e96163313e063140ac43548a3c
Uploaded "henry-the-anth.html" to address 8ad73285c6e3a6c7a2e1a2f3d359cecd99b84470ee22f9d95e573b9701890d7d
Uploaded "bundle.js" to address 4f9dee9055bd3179ecf5eaf357b02f482b8004d40d377cf886fbf4df3964f8d8
Uploaded "bundle.css" to address 364bc51cc292d66e6fb959f2c55bd2f12e6e49388a877a2dc497b7bda285f6ff
Uploaded "index-ants.png" to address b1ccf5989cc287532cae5a98adabc03bfc2cd95368c09e41dd066335a5e2a8f9
Uploaded "favicon.png" to address 2393fc4b9b138ef532b2ca72aa356f7fa863bfee4c267b04835738e5d471b461
Uploaded "global-thermonuclear-war.html" to address 11247c6c6dbe3627d98368a929b9a97128cd5ae5ae394cab22985aa02694a5e6
Uploaded "tetris.html" to address 4e9dee9d0b3f864bb36df2730cb7ad3089e5847d03058f04042b44ad38ba6410
Uploaded "riddim-ant.jpeg" to address 83d14bb4df8c4963087343026692759a383b4917592c09ee5a25b3d9e51d8cf1
Among 25 chunks, found 22 already existed in network, uploaded the leftover 3 chunks in 3 minutes 52 seconds
**************************************
*          Payment Details           *
**************************************
Made payment of NanoTokens(3) for 3 chunks
Made payment of NanoTokens(3) for royalties fees
New wallet balance: 0.000000094

The rsync command used to send the files from laptop to VPS was:

rsync -at --stats --delete $UPLOAD_DIR $VPS:$TMP_DIR

safe -V on both VPS and laptop both give:
sn_cli 0.95.0

Notice also that the VPS says those files amount to 76 chunks, while the second upload says there are 25 chunks. This is probably because I had previously attempted to upload from the laptop, so presumably it is saying 25 chunks remain to be uploaded of a total of 76. I think that should be made clear in the output.

[maqi]

Just notice in your second upload attempt, three files got said to be uploaded twice, and this didn't happen in your first upload attempt.
I guess the into 25 chunks is only about the new folder contains that three duplicated files, and the 3 chunks paid is regarding the metadata related to that 3 duplicated files ?

[happybeing]

I may not understand what you mean by metadata.

In the second case I am not generating and storing any additional metadata at that stage. I'm calling the sn_client files uploader, so I am not storing any additional metadata.

If you mean the metadata for the website, that occurs later (and is only a single chunk anyway).