Add CLI option to use the s3 file enlisting instead of the modified date to create daily/monthly files #70
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I could not fully test the concept due to access rights, but I could check the reading part of the public bucket. This PR adds an option
path-s3-folder
to recreate all daily/monthlmy files within a given S3 path, e.g.vph5_to_vpts --path-s3-folder uva/hdf5/nldhl/2019
. It scans all folder recursively for h5 files and uses this as input to define the days to recreate vpts files.@peterdesmet certainly open to a better name for the new CLI option :-)