Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement shadow rules for some rules (e.g. repeatmodeler) #3

Open
verku opened this issue Mar 1, 2022 · 2 comments
Open

Implement shadow rules for some rules (e.g. repeatmodeler) #3

verku opened this issue Mar 1, 2022 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@verku
Copy link
Collaborator

verku commented Mar 1, 2022

Shadow rules result in each execution of the rule to be run in isolated temporary directories. This “shadow” directory contains symlinks to files and directories in the current workdir. This is useful for running programs that generate lots of unused files which you don’t want to manually cleanup in your snakemake workflow. It can also be useful if you want to keep your workdir clean while the program executes, or simplify your workflow by not having to worry about unique filenames for all outputs of all rules.
shadow: "minimal" symlinks the inputs to the rule.
Once the rule successfully executes, the output file will be moved if necessary to the real path as indicated by output.
Shadow directories are stored one per rule execution in .snakemake/shadow/, and are cleared on successful execution. Consider running with the --cleanup-shadow argument every now and then to remove any remaining shadow directories from aborted jobs. The base shadow directory can be changed with the --shadow-prefix command line argument.

@verku verku added the enhancement New feature or request label Mar 1, 2022
@verku verku self-assigned this Mar 1, 2022
@verku
Copy link
Collaborator Author

verku commented Jul 14, 2022

Check if behaviour is desired on HPC cluster

@verku
Copy link
Collaborator Author

verku commented Oct 9, 2023

The mapping rule(s) where bam files are sorted create temporary files in the main Snakemake workflow directory that remain if the job fails. This could also be solved by using shadow rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant