NameError when using coolfiles on the server

[DaydreamingElephant]

Hello! Dry-run of quaich when using local coolfiles fails with the following error:
NameError in file /home/user/quaich/workflow/Snakefile, line 109:
name 'clr' is not defined
File "/home/user/quaich/workflow/Snakefile", line 109, in
ERROR:snakemake.logging:NameError in file /home/user/quaich/workflow/Snakefile, line 109:
name 'clr' is not defined
This has been observed on the latest version of quaich, where the test case ran successfully.
The coolfiles were produced by the recent version of the distiller pipeline.
It happens without the regards of actual position of the file - even in the inputs_folder/coolers in the quaich directory - or if we use relative or complete path.
Steps to replicate - write file path into samples.tsv, delete two other .tsv files without changing any configs.

[Phlya]

Mh I thought I had fixed this already... Try now? Just pushed a fix I think.

[DaydreamingElephant]

Thanks for a rapid responce! This error now eliminated. Now transitions to another error!
ERROR:snakemake.logging:FileNotFoundError in file /home/user/quaich/workflow/Snakefile, line 109:
[Errno 2] Unable to open file (unable to open file: name = 'test_cool', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
As previously mentioned, tried with file positioned both in its original directory and copied to inputs/coolers
Steps to replicate - write file path into samples.tsv, delete two other .tsv files without changing any configs.

[Phlya]

OK, I'd say for now you can just comment out that whole for loop, it's really not essential... The easiest solution at the moment.

[Phlya]

I have to say, I have a lot of changes planned in this branch: https://github.com/open2c/quaich/tree/pentads-sample-grouping
That's what I have been working on recently and it should work well, and have more functionality. But I haven't modified the test yet to work with it, so it's not merged into master.

[DaydreamingElephant]

Hello and thanks for the advice to check other branch! Swapping .keys for .values in line 106 (at the start of the loop) worked. It was kinda obvious in hindsight and I shouldn't have bothered you. Additionally, I would like to give a little feedback. I think there should be an explicit switch in config file that you use to declare whether you will use additional annotation files or not.

[Phlya]

Good catch!

Thanks, indeed the mechanism for configuring that is so far missing... It has already been suggested to look into it in one of the earlier issues, but I haven't implemented anything yet.

[DaydreamingElephant]

I just used empty dummy files so that the pipeline would not complain. Maybe mention in readme that annotations.tsv should contain at least one non-empty line, and if you don't need annotation - point it to an empty file?

[Phlya]

For now I just switch off pileups, since the annotations are only used for them... And leave the file as it is. And in the new branch mentioned above this doesn't stop TAD/loop pileups from being created.

But for sure I'll make it more user-friendly for this in the future!

[DaydreamingElephant]

Found another interesting tidbit: if one of the samples isn't named test_cool dry-run fails with this message:
InputFunctionException in file /home/user/quaich/workflow/Snakefile, line 628:
Error:
KeyError: 'test_cool'
Wildcards:
sample=test_cool
resolution=10000
Traceback:
File "/home/user/quaich/workflow/Snakefile", line 630, in
That's not difficult to work around but baffling.

[Phlya]

Ummm I'm not sure what you mean, I've run this many times with samples with other names... You just need to be consistent in naming across the config and files. For example, if you have the call_TADs section enabled, it'll expect, by default, a sample named test_cool.

[DaydreamingElephant]

Oh I see. I dang goofed up there, haven't I