caveman estep 1.12.0

[OCS-Consult]

This error has been reported before

[ERROR] (src/output.c: output_vcf_header:302 errno: No such file or directory) Error (0) writing contigs.
[ERROR] (src/estep.c: estep_main:517 errno: None) Error writing header to muts file.

The thread was closed suspecting a zlib error (fixed @ 1.2.8)
However we have the same error and this is on Centos 7 with zlib 1.2.7
$ cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)
and currently supports zlib:-
[root@server213-171-196-65 /]# rpm -qa zlib*
zlib-1.2.7-17.el7.x86_64
zlib-1.2.7-17.el7.i686
zlib-devel-1.2.7-17.el7.x86_64

As far a I understand there isn't a version of zlib 1.2.8 available for Centos 1.2.7.

Any advice will be appreciated.

[ghost]

zlib >= 1.2.3.5 should be sufficient given the functionality used.
Are you getting the correct zlib version linked used when you run
ldd caveman?

[OCS-Consult]

Hi David I got my C coder to fix the issue in zlib I believe. There is a limit to the chunck size the utility can use. We have revised the code to write out in smaller chunks and that works ok so far. I am just waiting on the users to qualify the VCF files we have produced. Will update the forum with the full fix in due course. Best Rgds Colin

[ChristopheLegendre]

@drjsanger
Hi David,

I have the same error for the caveman's step "estep". The error message is:
caveman estep -i 1
[ERROR] (src/output.c: output_vcf_header:302 errno: No such file or directory) Error (0) writing contigs. [ERROR] (src/estep.c: estep_main:518 errno: None) Error writing header to muts file.

My system is CentOS 7.2 with default gcc coming with that centOS version. The default libz is 1.2.7.
I forgot to mention that I am trying to use the latest version of caVEMan, i.e. 1.13.2 .

Despite version 1.2.7> 1.2.3.5 (the minimum version), caveman raised the error with libz version 1.2.7.
So I installed the latest version of zlib and it looks like it has been linked correctly to caveman (see below) after I ran the command " ./setup . " to install caveman with local htslib version provided by your script.

ldd /tools/CaVEMan/bin/caveman
linux-vdso.so.1 => (0x00007ffdce7d4000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcfc979a000) libz.so.1 => /tools/libz/1.2.11/lib/libz.so.1 (0x00007fcfc957e000) libm.so.6 => /lib64/libm.so.6 (0x00007fcfc927c000) libdl.so.2 => /lib64/libdl.so.2 (0x00007fcfc9078000) libc.so.6 => /lib64/libc.so.6 (0x00007fcfc8cb4000) /lib64/ld-linux-x86-64.so.2 (0x00007fcfc99de000)

I do not know C programming, but it seems that functions related to libz are not recognized, such as gzprintf, gzopen

Questions:

• Is there any other library that should be linked to caveman?
• Should I install the version 1.2.3.5 for libz and link caveman to it?
• Would it be possible to get static binaries for caveman?

Thanks in advance for your help,
Best,
Chris

[ghost]

@ChristopheLegendre I have just tested this using centos7 and it worked for me. Can I ask what reference you're using please?

[ChristopheLegendre]

@drjsanger
Sure.
The reference Genome is hs37d5 (GRCh37).
The contigs are not prefixed by "chr".
Contigs list attached.
hs37d5.fa.chrom.contigs.txt

[ghost]

Thank you @ChristopheLegendre .

A quick update:

I have tested with a clean install of Centos 7

$ cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

Using yum installed libraries only:

$ ldd install/bin/caveman
	linux-vdso.so.1 =>  (0x00007ffc61ed3000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f58f7fe5000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f58f7dcf000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f58f7acc000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f58f78c8000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f58f74fb000)
	/lib64/ld-linux-x86-64.so.2 (0x000055c64e6a5000)

Which leads to CaVEMan using zlib version 1.2.7

$ ls -l /lib64/libz.so.1
lrwxrwxrwx. 1 root root 13 Mar  2  2018 /lib64/libz.so.1 -> libz.so.1.2.7

This is failing to reproduce the error when using GRCh37d5 (including all contigs to ensure largest length header possible).

It looks from your ldd output you have libz 1.2.11 installed and linked.

I'll try that next.

Dave

[ghost]

I can't replicate this error using zlib 1.2.11.

I've noticed your GRCh37 has more contigs than ours. I'll try this again using GRCh38 as that has even more contigs to push to limit of the zlib buffer setting and see that is what's causing the problems. Since @OCS-Consult reported that changing the zlib buffer was fixing things I am suspecting a lot of contigs is growing to large for the write buffer.

I'd like to reproduce it before I go ahead and make a hotfix though. Still working on it.

Dave

[ghost]

I have now successfully reproduced using a larger number of contigs (GRCh38).

Centos 7.4
zlib 1.2.11

I'll investigate how modifying the buffer size for zlib after file opening can correct this.

[ChristopheLegendre]

Hi David,

Thanks for investigating.
For information, the extra contigs we added to our reference are very small contigs such the ERCC-xxxx, EBV and some ribosomal ones.

[ghost]

I've added a dynamic calculation of contig header section size to ensure the gzbuffer is now set correctly.

1.13.3 contains this fix. @OCS-Consult
apologies this took so long to get fixed!

[ChristopheLegendre]

@drjsanger
Hi David,

No problems. I am glad I could give some feedback.
Thanks for updating caveman.
I have tested it and got errors; I opened a new issue #82
Before running my newcaveman 1.13.3 executable, I checked and it is well linked to libz 1.2.11.

Best,
Chris

[OCS-Consult]

No problem David – you probably hit the issues we fixed with zlib…☺ I’ll let my chaps know. Rgds Colin From: David Jones [mailto:[email protected]] Sent: 07 September 2018 16:32 To: cancerit/CaVEMan Cc: Colin Fairbairn; Mention Subject: Re: [cancerit/CaVEMan] caveman estep 1.12.0 (#77) I've added a dynamic calculation of contig header section size to ensure the gzbuffer is now set correctly. 13.3.3<https://github.com/cancerit/CaVEMan/releases/tag/1.13.3> contains this fix. @OCS-Consult<https://github.com/OCS-Consult> apologies this took so long to get fixed! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#77 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AiEfrlIwOuTdSOjNn7vBCEYU0OqJXCuDks5uYpFYgaJpZM4RpW7X>.

…

________________________________ DISCLAIMER : This email contains proprietary information some or all of which may be legally privileged. It is for the intended recipient only. If an addressing or transmission error has misdirected this email, please notify the author by replying to this email. If you are not the intended recipient you must not use, disclose, distribute, copy, print or relay on this email. Whilst OCS and its group of companies has taken precautions to avoid the transmission of viruses, it is the responsibility of the recipient to ensure that the onward transmission, opening or use of this message and any attachments will not adversely affect its system or data. No responsibility is accepted by OCS Consulting plc and its group of companies in this regard and the recipient should carry out such virus and other checks as it considers appropriate Registered Office: 66 Prescot Street, London E1 8NN. United Kingdom Registered No. 1894212 VAT Registration No. 421 3151 09