Issue70

[kopardev]

Changes

• adding requirements.txt for easy creation of environment in "spacesavers2" docker (fix chore: create requirements.txt #68, @kopardev)
• grubbers --limit can be < 1 GiB (float) fix grubbers does not allow --limit < 1 GiB #70
• grubbers has new --outfile argument.
• grubbers output file format changed. New original file column added..
• blamematrix has now been moved into mimeo.
• mimeo --duplicateonly logic fix -z the only duplicates option for mimeo is not correctly implemented. #71
• mimeo files.gz always includes the original file as the first one in the filelist.
• mimeo now has kronatools compatible output. ktImportText is also run if in PATH to generate HTML report for duplicates only. (fix Krona charts #46, @kopardev)
• e2e overhauled, improved and well commented.
• documentation updated.
• total size now closely resemble df results fix Total sizes of folders do not add up #75
• ls size is not "size" and du size is called "calculated_size" in the new FileDetails object.
• files with future timestamps are handles correctly (fix Bug with files with future timestamp #76, @kopardev)

[kelly-sovacool]

Currently, the output files from catalog, grubbers, and mimeo all start with a common prefix to encode the date, input directory, and user. However, the blame matrix is always called blamematrix.tsv -- should this file instead be renamed to include the same common prefix as the other files?

[kelly-sovacool]

I fixed an issue in the e2e docs (bdee6d3) and also made minor updates to the changelog notes -- e.g. succinctly describing issue 71 so readers can get the gist of the fix without going to that issue.

[kelly-sovacool]

I ran spacesavers2_e2e on a few different directories. Results all look good, especially liking the krona reports.

I ran spacesavers2_grubbers --limit 0 separately, as none of the directories I tested had duplicate files over the default 5 GiB. It worked as expected.

I wonder if e2e should have also have a limit argument to pass on to grubbers with the same default value?

[kopardev]

I wonder if e2e should have also have a limit argument to pass on to grubbers with the same default value?

It does

[kopardev]

I fixed an issue in the e2e docs (bdee6d3) and also made minor updates to the changelog notes -- e.g. succinctly describing issue 71 so readers can get the gist of the fix without going to that issue.

Further updated docs (f72923a)

[kopardev]

Currently, the output files from catalog, grubbers, and mimeo all start with a common prefix to encode the date, input directory, and user. However, the blame matrix is always called blamematrix.tsv -- should this file instead be renamed to include the same common prefix as the other files?

Done in 615f1ca .

[kopardev]

Also, fixes #64