Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to Nextclade v3 #281

Merged
merged 8 commits into from
Oct 30, 2024
Merged

Migrate to Nextclade v3 #281

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3850af5
ingest: Update to nextclade3
joverlee521 Oct 18, 2024
cbcc7d4
ingest: move Nextclade dataset to `data/` and outputs to `results/`
joverlee521 Oct 18, 2024
1234479
ingest: Rename Nextclade metadata fields with augur curate rename
joverlee521 Oct 18, 2024
54ca356
ingest: Merge Nextclade metadata with `augur merge`
joverlee521 Oct 18, 2024
475b64f
ingest/nextstrain_automation: Update `files_to_upload`
joverlee521 Oct 18, 2024
6b5153c
phylogenetic: Update description.md for Nextclade v3
joverlee521 Oct 18, 2024
42c1040
ingest/nextclade: Add log and benchmark
joverlee521 Oct 22, 2024
dab86ff
ingest/nextclade: Use all available workflow cores
joverlee521 Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ingest/rules/nextclade.smk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ rule run_nextclade:
# The lambda is used to deactivate automatic wildcard expansion.
# https://github.com/snakemake/snakemake/blob/384d0066c512b0429719085f2cf886fdb97fd80a/snakemake/rules.py#L997-L1000
translations=lambda w: "results/translations/{cds}.fasta",
threads: 4
threads: workflow.cores
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shaved off 8 mins for the Nextclade job, but only 3 mins off the total workflow.

Using all of the cores prevents Snakemake from scheduling other concurrent jobs such as upload_to_s3 jobs. The upload jobs can be slow as well, so we could adjust the threads to something like workflow.cores * 0.75.

It's only a couple minute difference so I'm not going to worry about it for now.

log:
"logs/run_nextclade.txt",
benchmark:
Expand Down