Saving more provenance information

[pvandyken]

The problem

This issue is somewhat relate to #182, but more specifically concerns what information is saved in the auto-generated snakemake.yml file. Currently we keep the entire config, any arguments passed to run.py (including snakemake args), and the snakemake and output directories. For full replicability, it would also be helpful to save the apṗ version (related to #85), and the snakemake version used to run the pipeline.

[akhanf]

I agree!

For the app version would we need to force users to expose the version in a particular way?

[pvandyken]

Yeah, so that's the tricky bit, there's no one correct way to do this. Of the options listed here, the pkg_resources approach is probably the most method agnostic.

One other tricky is mid-release development version. Other apps will use git hashes as part of the dev versions, but poetry doesn't handle this directly. One would probably need a git precommit hook with versioneer or similar. But also check out this: poetry-dynamic-versioning. I think I'll make a separate issue about it, it looks pretty cool.

[pvandyken]

One more related thought: so long as we're reconstructing the config file from scratch, and given the order of attributes is arbitrary, it might be nice to purposefully organize things (and even put comments in if that can be managed... not sure) to make the file as human-readable as possible.

[akhanf]

yes a more human-readable config would be great!