Feat/clin 2947 add exomiser #25

LysianeBouchard · 2024-09-18T18:53:55Z

This PR introduce the exomiser tool in the pipeline. Additionally, it introduces a new parameter called tools, which allows users to specify whether VEP, Exomiser, or both should be executed.

Additionally, there is some refactoring in the main workflow (postprocessing.nf) to use the def keyword and more standard variable names. I could not do it only for exomiser variables as, strangely, the code was not compiling when a channel with local scope was derived from a channel with non-local scope.

Here is the associated JIRA for exomiser:
https://ferlab-crsj.atlassian.net/browse/CLIN-2947

Limitations

For the schema, if-else blocks are used to conditionally require exomiser/vep parameters based on the value of the tools parameter.

If an exomiser or vep parameter is removed/added/modified, the logic in the if-else block will need to be manually updated. For example, if the removed parameter was required, it must be manually removed from the list of required parameters.

Tests

Running the pipeline:

Run the pipeline locally with the public test dataset (V3).
- Check that vep output files are under results/vep and exomiser output files under results/exomiser/results.
- Check that vep and exomiser output files are non empty.
Run the pipeline locally with the public test dataset (V3) again, this time commenting in the test config parameters exomiser_analysis_wes and exomiser_analysis_wgs. We want to see if the pipeline is correctly executed using the default analysis files.
Run the pipeline on juno with the public test dataset (V3).
Make sure that ci tests pass

Note: the V3 dataset covers REMM and CADD data sources and produces non empty exomiser files.

Linter:

Run the linter locally and compare with the main branch

Test the schema:

Check that the nf-core schema build commands still works and does not break the added if-else blocks.
nf-core schema lint
nf-core schema docs
nf-core schema validate . data-test/paramsJuno.json
nf-core schema build

The validate command seem to take into account if-else blocks correctly according to my tests.
I tweaked the paramsJuno.json file to remove a required exomiser parameter and it reported it correctly.

Test exomiser/vep parameter validation (stub mode)

exomiser tool present:
- missing required parameter: should be detected and pipeline should stop
- missing optional parameter (ex: cadd version): pipeline should run with exomiser
- remm/cadd version specified:
  - missing other remm/cadd parameters: missing parameters should be detected and pipeline should stop
  - all remm/cadd paremeters specified: pipeline should run exomiser and pass remm/cadd options
- remm/cadd version not specified:
  - missing other remm/cadd parameters: pipeline should run exomiser and not pass remm/cadd options
- exomiser_analysis_wes / exomiser_analysis_wgs not specified: pipeline should use the default analysis files
exomiser tool absent:
- missing required/optional exomiser parameter: pipeline should run without exomiser
- remm/cadd versions specified:
  - missing other remm/cadd parameters: pipeline should run without exomiser
vep tool present:
- missing required vep parameter: should be detected and pipeline should stop
vep tool absent:
- missing required/optional vep parameters: pipeline should run without vep
check that exomiser is run if tools is set to vep,exomiser | exomiser,vep | exomiser
check that exomiser is skipped if tools is set to vep | empty string | null
check that vep is run if tools is set to vep,exomiser | exomiser,vep | vep
check that vep is skipped if tools is set to exomiser | empty string | null

Extra tests with CADD/REMM

If cadd/remm parameters are not passed (i.e. missing cadd/remm version) and the analysis file includes cadd/remm pathogenicity sources, the pipeline should be executed, but it should fail at the exomiser step
if cadd/remm parameters are passed and the analysis file does not include cadd/remm pathogenicity sources, the pipeline should be executed successfully

Validate json format is supported for analysis files and familyPheno files:

ask copilot to generate a json copy of one of the default analysis file and pass it at the command line
ask copilot to generate a json copy of one of the family file, create a copy of the test sample sheet pointing on the json family file and run the pipeline locally with it

PR checklist

workflows/postprocessing.nf

FelixAntoineLeSieur · 2024-09-23T20:34:59Z

assets/schema_input.json

+                "errorMessage": "Filename of the pedigree file, mandatory for exomiser",
+                "meta": ["familypheno"],
+                "format": "file-path",
+                "pattern": "^\\S*.y(a)?ml$",


on acceptait les json aussi?

Ah oui c'est vrai bon point. Will do.

Also included a pattern for the analysis files in the parameters schema.

docs/reference_data.md

README.md

DamienGnst · 2024-09-25T17:42:32Z

README.md

@@ -140,11 +95,10 @@ The documentation of the various tools used in this workflow are available here:



Remove last parenthesis + update link

HardFiltering

docs/reference_data.md

DamienGnst · 2024-09-25T17:58:24Z

docs/reference_data.md

+  - 1000G reference file : 1000G_phase1.snps.high_confidence.hg38.vcf.gz
+  - SNP database : Homo_sapiens_assembly38.dbsnp138.vcf.gz
+
+## Reference Genome


I think we can put this part in first before VQSR no?

Sure, good point

docs/reference_data.md

This commit introduces the exomiser tool in the pipeline. Additionally, it introduces a new parameter called `tools`, which allows users to specify whether VEP, Exomiser, or both should be executed. Other changes: - Group vep output files (i.e. vcf + .tbi files) in subfolder `vep`. - Improve documentation (README, OUTPUT, USAGE, REFERENCE_DATA). - Refactor postprocessing workflow to use `def` for local variables and standardize variable names. - Update GitHub CI nf-test command: remove local tag constraint and activate CI mode. - Replace workflow diagram in README.

LysianeBouchard requested review from FelixAntoineLeSieur, dmorais and DamienGnst as code owners September 18, 2024 18:53

LysianeBouchard marked this pull request as draft September 18, 2024 18:54

LysianeBouchard force-pushed the feat/CLIN-2947-add-exomiser branch 26 times, most recently from 9eaf4fe to 860ac7f Compare September 23, 2024 15:59

LysianeBouchard force-pushed the feat/CLIN-2947-add-exomiser branch 9 times, most recently from 81c6414 to acb6e8f Compare September 23, 2024 19:44

FelixAntoineLeSieur reviewed Sep 23, 2024

View reviewed changes

workflows/postprocessing.nf Show resolved Hide resolved

FelixAntoineLeSieur reviewed Sep 23, 2024

View reviewed changes

LysianeBouchard force-pushed the feat/CLIN-2947-add-exomiser branch 6 times, most recently from ae7818e to a143836 Compare September 24, 2024 19:24

DamienGnst reviewed Sep 24, 2024

View reviewed changes

docs/reference_data.md Outdated Show resolved Hide resolved

docs/reference_data.md Show resolved Hide resolved

DamienGnst reviewed Sep 25, 2024

View reviewed changes

DamienGnst reviewed Sep 26, 2024

View reviewed changes

docs/reference_data.md Show resolved Hide resolved

LysianeBouchard force-pushed the feat/CLIN-2947-add-exomiser branch 2 times, most recently from 05fe1e9 to 5ba7519 Compare September 26, 2024 13:58

LysianeBouchard force-pushed the feat/CLIN-2947-add-exomiser branch from 6aaee05 to 4a2a0f4 Compare October 1, 2024 18:50

LysianeBouchard marked this pull request as ready for review October 1, 2024 20:07

DamienGnst approved these changes Oct 1, 2024

View reviewed changes

LysianeBouchard merged commit 85af2f2 into main Oct 1, 2024
3 checks passed

LysianeBouchard deleted the feat/CLIN-2947-add-exomiser branch October 1, 2024 20:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/clin 2947 add exomiser #25

Feat/clin 2947 add exomiser #25

LysianeBouchard commented Sep 18, 2024 •

edited

Loading

FelixAntoineLeSieur Sep 23, 2024

LysianeBouchard Sep 24, 2024

LysianeBouchard Sep 24, 2024

LysianeBouchard Sep 24, 2024

DamienGnst Sep 25, 2024

LysianeBouchard Sep 26, 2024

DamienGnst Sep 25, 2024

LysianeBouchard Sep 26, 2024

LysianeBouchard Sep 26, 2024

		@@ -140,11 +95,10 @@ The documentation of the various tools used in this workflow are available here:

Feat/clin 2947 add exomiser #25

Feat/clin 2947 add exomiser #25

Conversation

LysianeBouchard commented Sep 18, 2024 • edited Loading

PR checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LysianeBouchard commented Sep 18, 2024 •

edited

Loading