Launch directory dependent FileNotFound #225
Comments
|
Hi @alkaZeltser, I'm not an expert with Azure but I wonder if any general answers for Nextflow x Azure might be in the their docs: https://www.nextflow.io/docs/latest/azure.html? Also here: https://microsoft.github.io/Genomics-Community/mydoc_nextflow.html |
|
I think the successful run error might be caused by the For the unsuccessful run, are you using relative or absolute (starts with |
|
Hi folks, thank you for your comments. I've been poking around a bit more, and I'm pretty sure that the issue is a result of nextflow to docker mounting procedures. Nextflow will mount the working directory and the directories of any explicit inputs (as provided in the nextflow run command) but the path provided in the sample sheet is not explicitly mounted. It is only accessible if it happens to be in a directory that is encompassed by one of the explicit mounts. In some cases this is not difficult for me to accommodate, but I have encountered a case where my genomic data files are sitting too far away. I attempted a sym-link but unfortunately that didn't work. Do you have any tips for modifying nextflow configs to allow additional mounts? |
You were right, removing the email worked to silence the |
Adapting this singularity example might work This should mount the directory from your host filesystem to the same place inside the docker container. |
I've found a solution that works well, but with a necessary modification to the suggestion, so I felt it was worth documenting some details in case it is useful to anyone else. To recap the problem:
Suggested solution: add input fileset path to global docker Contents of While this satisfies The error is a Here's the final solution we settled on: This worked, and I was able to complete a full run of pgsc_calc.
I think the main issue here is still the fact that |
|
Thanks for documenting your fix 😄 I'm glad it's working for you.
That's an interesting idea I hadn't thought of. Nextflow is responsible for configuring and orchestrating the containers, so the CLI applications aren't able to edit the containers they're running inside. I'll investigate and see if I can figure out a workaround #240 |
Description of the bug
My input VCF specified through the sample_sheet.csv is not recognized (FileNotFound/ file doesn't exist from
pgscatalog_utils.samplesheet.check) when the tool is launched from some directories, but IS recognized when the tool launched from other directories. I discovered this by chance when attempting to debug theFileNotFoundI encountered on my first attempt at running the tool, I just so happened to launch it from a different directory without having changed anything else.To be clear,
launchDirandworkDirchange, but theinputdir where sample sheet is stored and path to VCF do NOT change.I have read and write permissions in all of these directories:
I suspect this is some sort of mounting issue related to my Azure HPC environment and the docker platform, but I unfortunately don't have the expertise to troubleshoot further. It is perhaps worth noting that pgsc_calc was stored by default under $HOME which is in a different mount in our HPC from where the data are stored and the tools is launched.
Note, in the successful launches the tool seems to complete all relevant workflows but also fails to exit cleanly with:
I think this is a separate issue though.
Command used and terminal output
Relevant files
System information
The text was updated successfully, but these errors were encountered: