Libcrypto.so.1.1 not available?

[him1532]

I am seeing an error while running kallisto sleuth pipline.

Error in group sleuth-init:
jobs:
rule sleuth_init:
jobid: 2
output: results/sleuth/model_X.rds, results/sleuth/model_X.designmatrix.rds
log: logs/sleuth/model_X.init.log (check log file(s) for error details)
rule compose_sample_sheet:
jobid: 9
output: results/sleuth/model_X.samples.tsv
log: logs/model_X.compose-sample-sheet.log (check log file(s) for error details)

Error executing group job sleuth-init on cluster (jobid: 9159f0df-40ec-5076-ac29-7ac8600dd138, external: Your job 1509748 ("snakejob.sleuth-init.9159f0df-40ec-5076-ac29-7ac8600dd138.sh") has been submitted, jobscript: /home/RNA/.snakemake/tmp.b2r32iv8/snakejob.sleuth-init.9159f0df-40ec-5076-ac29-7ac8600dd138.sh). For error details see the cluster log and the log files of the involved rule(s).
Removing output files of failed job compose_sample_sheet since they might be corrupted:
results/sleuth/model_X.samples.tsv
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-18T142901.516512.snakemake.log

When I looked at the log files.

cat logs/sleuth/model_X.init.log
Error: package or namespace load failed for ‘sleuth’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object >'/home/RNA/.snakemake/conda/e9ae8cf1b054539d07bb34138d5bde75_/lib/R/library/rhdf5/libs/rhdf5.so':
libcrypto.so.1.1: cannot open shared object file: No such file or directory
Execution halted

Do I need to install some other library?
Thank you.

---

Following is the output of snakemake/log/2023-12-18T142901.516512.snakemake.log

Workflow defines that rule get_transcriptome is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule get_annotation is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule get_transcript_info is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule convert_pfam is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule calculate_cpat_hexamers is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule calculate_cpat_logit_model is eligible for caching between workflows (use the --cache argument to enable this).
Workflow defines that rule get_spia_db is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Your conda installation is not configured to use strict channel priorities. This is however crucial for having robust and correct environments (for details, see https://conda-forge.org/docs/user/tipsandtricks.html). Please consider to configure strict priorities by executing 'conda config --set channel_priority strict'.
Using shell: /usr/bin/bash
Provided cluster nodes: 1
Singularity containers: ignored
Job stats:
job count

---

all 1
compose_sample_sheet 2
cutadapt_pe 2
diffexp_datavzrd 1
get_transcript_info 1
get_transcriptome 1
ihw_fdr_control 3
kallisto_index 1
kallisto_quant 2
logcount_matrix 1
plot_bootstrap 1
plot_diffexp_heatmap 1
plot_diffexp_pval_hist 3
plot_fragment_length_dist 2
plot_group_density 1
plot_pca 1
render_datavzrd_config_diffexp 1
sleuth_diffexp 1
sleuth_init 2
vega_volcano_plot 1
total 29

Select jobs to execute...

[Mon Dec 18 14:29:11 2023]
rule cutadapt_pe:
input: /RNAs/RNAseq_test_PPM1D_231201_N12/EOL-1_GSK_1.fq.gz, /RNAs/RNAseq_test_PPM1D_231201_N12/EOL-1_GSK_2.fq.gz
output: results/trimmed/EOL-2-1.1.fastq.gz, results/trimmed/EOL-2-1.2.fastq.gz, results/trimmed/EOL-2-1.qc.txt
log: results/logs/cutadapt/EOL-2-1.log
jobid: 8
reason: Missing output files: results/trimmed/EOL-2-1.2.fastq.gz, results/trimmed/EOL-2-1.1.fastq.gz
wildcards: sample=EOL-2, unit=1
threads: 8
resources: mem_mb=16486, mem_mib=15723, disk_mb=16486, disk_mib=15723, tmpdir=

Submitted job 8 with external jobid 'Your job 1509741 ("snakejob.cutadapt_pe.8.sh") has been submitted'.
[Mon Dec 18 14:33:01 2023]
Finished job 8.
1 of 29 steps (3%) done
Select jobs to execute...

[Mon Dec 18 14:33:01 2023]
rule cutadapt_pe:
input: /RNAs/RNAseq_test_PPM1D_231201_N12/EOL-1_DMSO_1.fq.gz, /RNAs/RNAseq_test_PPM1D_231201_N12/EOL-1_DMSO_2.fq.gz
output: results/trimmed/EOL-1-1.1.fastq.gz, results/trimmed/EOL-1-1.2.fastq.gz, results/trimmed/EOL-1-1.qc.txt
log: results/logs/cutadapt/EOL-1-1.log
jobid: 4
reason: Missing output files: results/trimmed/EOL-1-1.2.fastq.gz, results/trimmed/EOL-1-1.1.fastq.gz
wildcards: sample=EOL-1, unit=1
threads: 8
resources: mem_mb=15400, mem_mib=14687, disk_mb=15400, disk_mib=14687, tmpdir=

Submitted job 4 with external jobid 'Your job 1509742 ("snakejob.cutadapt_pe.4.sh") has been submitted'.
[Mon Dec 18 14:39:51 2023]
Finished job 4.
2 of 29 steps (7%) done
Select jobs to execute...

[Mon Dec 18 14:39:51 2023]
rule get_transcript_info:
output: resources/transcript-info.rds
log: logs/get_transcript_info.log
jobid: 10
reason: Missing output files: resources/transcript-info.rds
resources: mem_mb=1000, mem_mib=954, disk_mb=1000, disk_mib=954, tmpdir=

Submitted job 10 with external jobid 'Your job 1509743 ("snakejob.get_transcript_info.10.sh") has been submitted'.
[Mon Dec 18 14:44:41 2023]
Finished job 10.
3 of 29 steps (10%) done
Select jobs to execute...

[Mon Dec 18 14:44:41 2023]
rule get_transcriptome:
output: resources/transcriptome.cdna.fasta
log: logs/get-transcriptome/cdna.log
jobid: 6
reason: Missing output files: resources/transcriptome.cdna.fasta
wildcards: type=cdna
resources: mem_mb=1000, mem_mib=954, disk_mb=1000, disk_mib=954, tmpdir=

Submitted job 6 with external jobid 'Your job 1509744 ("snakejob.get_transcriptome.6.sh") has been submitted'.
[Mon Dec 18 14:47:42 2023]
Finished job 6.
4 of 29 steps (14%) done
Select jobs to execute...

[Mon Dec 18 14:47:42 2023]
rule kallisto_index:
input: resources/transcriptome.cdna.fasta
output: results/kallisto_cdna/transcripts.cdna.idx
log: results/logs/kallisto_cdna/index.cdna.log
jobid: 5
reason: Missing output files: results/kallisto_cdna/transcripts.cdna.idx; Input files updated by another job: resources/transcriptome.cdna.fasta
resources: mem_mb=1000, mem_mib=954, disk_mb=1000, disk_mib=954, tmpdir=

Submitted job 5 with external jobid 'Your job 1509745 ("snakejob.kallisto_index.5.sh") has been submitted'.
[Mon Dec 18 14:54:22 2023]
Finished job 5.
5 of 29 steps (17%) done
Select jobs to execute...

[Mon Dec 18 14:54:22 2023]
rule kallisto_quant:
input: results/trimmed/EOL-2-1.1.fastq.gz, results/trimmed/EOL-2-1.2.fastq.gz, results/kallisto_cdna/transcripts.cdna.idx
output: results/kallisto_cdna/EOL-2-1
log: results/logs/kallisto_cdna/quant/EOL-2-1.log
jobid: 7
reason: Missing output files: results/kallisto_cdna/EOL-2-1; Input files updated by another job: results/kallisto_cdna/transcripts.cdna.idx, results/trimmed/EOL-2-1.2.fastq.gz, results/trimmed/EOL-2-1.1.fastq.gz
wildcards: sample=EOL-2, unit=1
threads: 5
resources: mem_mb=20797, mem_mib=19834, disk_mb=20797, disk_mib=19834, tmpdir=

Submitted job 7 with external jobid 'Your job 1509746 ("snakejob.kallisto_quant.7.sh") has been submitted'.
[Mon Dec 18 15:20:13 2023]
Finished job 7.
6 of 29 steps (21%) done
Select jobs to execute...

[Mon Dec 18 15:20:13 2023]
rule kallisto_quant:
input: results/trimmed/EOL-1-1.1.fastq.gz, results/trimmed/EOL-1-1.2.fastq.gz, results/kallisto_cdna/transcripts.cdna.idx
output: results/kallisto_cdna/EOL-1-1
log: results/logs/kallisto_cdna/quant/EOL-1-1.log
jobid: 3
reason: Missing output files: results/kallisto_cdna/EOL-1-1; Input files updated by another job: results/kallisto_cdna/transcripts.cdna.idx, results/trimmed/EOL-1-1.2.fastq.gz, results/trimmed/EOL-1-1.1.fastq.gz
wildcards: sample=EOL-1, unit=1
threads: 5
resources: mem_mb=19758, mem_mib=18843, disk_mb=19758, disk_mib=18843, tmpdir=

Submitted job 3 with external jobid 'Your job 1509747 ("snakejob.kallisto_quant.3.sh") has been submitted'.
[Mon Dec 18 15:37:44 2023]
Finished job 3.
7 of 29 steps (24%) done
Select jobs to execute...
[Mon Dec 18 15:37:45 2023]

group job sleuth-init (jobs in lexicogr. order):

[Mon Dec 18 15:37:45 2023]
rule compose_sample_sheet:
input: config/samples.tsv, config/units.tsv, results/kallisto_cdna/EOL-1-1, results/kallisto_cdna/EOL-2-1
output: results/sleuth/model_X.samples.tsv
log: logs/model_X.compose-sample-sheet.log
jobid: 9
reason: Missing output files: results/sleuth/model_X.samples.tsv; Input files updated by another job: results/kallisto_cdna/EOL-1-1, results/kallisto_cdna/EOL-2-1
wildcards: model=model_X
resources: mem_mb=1000, mem_mib=954, disk_mb=1000, disk_mib=954, tmpdir=

[Mon Dec 18 15:37:45 2023]
rule sleuth_init:
input: results/kallisto_cdna/EOL-1-1, results/kallisto_cdna/EOL-2-1, results/sleuth/model_X.samples.tsv, resources/transcript-info.rds
output: results/sleuth/model_X.rds, results/sleuth/model_X.designmatrix.rds
log: logs/sleuth/model_X.init.log
jobid: 2
reason: Missing output files: results/sleuth/model_X.rds; Input files updated by another job: resources/transcript-info.rds, results/kallisto_cdna/EOL-1-1, results/sleuth/model_X.samples.tsv, results/kallisto_cdna/EOL-2-1
wildcards: model=model_X
threads: 6
resources: mem_mb=, disk_mb=, tmpdir=

Submitted group job 9159f0df-40ec-5076-ac29-7ac8600dd138 with external jobid 'Your job 1509748 ("snakejob.sleuth-init.9159f0df-40ec-5076-ac29-7ac8600dd138.sh") has been submitted'.
[Mon Dec 18 15:38:04 2023]
Error in group sleuth-init:
jobs:
rule sleuth_init:
jobid: 2
output: results/sleuth/model_X.rds, results/sleuth/model_X.designmatrix.rds
log: logs/sleuth/model_X.init.log (check log file(s) for error details)
rule compose_sample_sheet:
jobid: 9
output: results/sleuth/model_X.samples.tsv
log: logs/model_X.compose-sample-sheet.log (check log file(s) for error details)

Error executing group job sleuth-init on cluster (jobid: 9159f0df-40ec-5076-ac29-7ac8600dd138, external: Your job 1509748 ("snakejob.sleuth-init.9159f0df-40ec-5076-ac29-7ac8600dd138.sh") has been submitted, jobscript: /home/RNA/.snakemake/tmp.b2r32iv8/snakejob.sleuth-init.9159f0df-40ec-5076-ac29-7ac8600dd138.sh). For error details see the cluster log and the log files of the involved rule(s).
Removing output files of failed job compose_sample_sheet since they might be corrupted:
results/sleuth/model_X.samples.tsv
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-18T142901.516512.snakemake.log

[dlaehnemann]

We have seen this ourselves, this was down to an undocumented implicit dependency in an upstream package that can mess up linking of curl. We fixed this for newer versions of the respective bioconda packages, but the changes pulling this into this workflow are still unmerged (and thus unreleased). They are in pull request #77. As a dirty fix to get past this, you could change the tag= entry in the github() function in the module import statement in your Snakefile to instead point to a branch=, using the branch behind that pull request:

github(..., branch="fix-canonical-transcript-mapped-read-extraction")`

However, this will stop working once we merge the pull request. So you will then have to switch back to the latest release. I'll try to remember to ping you here, once this is merged and released.

[him1532]

ok. got it. will try and report back.

[him1532]

I made changes to the Snakefiles as you suggested.

This time I have following error.

Error in group sleuth-init:
jobs:
rule compose_sample_sheet:
jobid: 29
output: results/sleuth/model_X.samples.tsv
log: logs/model_X.compose-sample-sheet.log (check log file(s) for error details)
rule sleuth_init:
jobid: 2
output: results/sleuth/model_X.rds, results/sleuth/model_X.designmatrix.rds
log: logs/sleuth/model_X.init.log (check log file(s) for error details)

Error executing group job sleuth-init on cluster (jobid: 9159f0df-40ec-5076-ac29-7ac8600dd138, external: Your job 1509776 ("snakejob.sleuth-init.9159f0df-40ec-5076-ac29-7ac8600dd138.sh") has been submitted, jobscript: /home/him/RNA/.snakemake/tmp.isl2w3r_/snakejob.sleuth-init.9159f0df-40ec-5076-ac29-7ac8600dd138.sh). For error details see the cluster log and the log files of the involved rule(s).
Removing output files of failed job compose_sample_sheet since they might be corrupted:
results/sleuth/model_X.samples.tsv

This logs/model_X.compose-sample-sheet.log has nothing in it.

While logs/model_X.compose-sample-sheet.log out put is below.

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.4 ✔ readr 2.1.4
✔ forcats 1.0.0 ✔ stringr 1.5.1
✔ ggplot2 3.4.4 ✔ tibble 3.2.1
✔ lubridate 1.9.3 ✔ tidyr 1.3.0
✔ purrr 1.0.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (http://conflicted.r-lib.org/) to force all conflicts to become errors
Rows: 12 Columns: 3
── Column specification ────────────────────────────────────────────────────────
Delimiter: "\t"
chr (3): sample, condition, path

ℹ Use spec() to retrieve the full column specification for this data.
ℹ Specify the column types or set show_col_types = FALSE to quiet this message.
Error in all_of():
! Can't subset columns that don't exist.
✖ Column batch_effect doesn't exist.
Backtrace:
▆

1. ├─samples %>% drop_na(c(sample, path, all_of(variables)))
2. ├─tidyr::drop_na(., c(sample, path, all_of(variables)))
3. ├─tidyr:::drop_na.data.frame(., c(sample, path, all_of(variables)))
4. │ └─tidyselect::eval_select(expr(c(!!!dots)), data, allow_rename = FALSE)
5. │ └─tidyselect:::eval_select_impl(...)
6. │ ├─tidyselect:::with_subscript_errors(...)
7. │ │ └─rlang::try_fetch(...)
8. │ │ └─base::withCallingHandlers(...)
9. │ └─tidyselect:::vars_select_eval(...)
10. │ └─tidyselect:::walk_data_tree(expr, data_mask, context_mask)
11. │ └─tidyselect:::eval_c(expr, data_mask, context_mask)
12. │ └─tidyselect:::reduce_sels(node, data_mask, context_mask, init = init)
13. │ └─tidyselect:::walk_data_tree(new, data_mask, context_mask)
14. │ └─tidyselect:::eval_c(expr, data_mask, context_mask)
15. │ └─tidyselect:::reduce_sels(node, data_mask, context_mask, init = init)
16. │ └─tidyselect:::walk_data_tree(new, data_mask, context_mask)
17. │ └─tidyselect:::eval_context(expr, context_mask, call = error_call)
18. │ ├─tidyselect:::with_chained_errors(...)
19. │ │ └─rlang::try_fetch(...)
20. │ │ ├─base::tryCatch(...)
21. │ │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
22. │ │ │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
23. │ │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
24. │ │ └─base::withCallingHandlers(...)
25. │ └─rlang::eval_tidy(as_quosure(expr, env), context_mask)
26. ├─tidyselect::all_of(variables)
27. │ └─tidyselect:::as_indices_impl(x, vars = vars, strict = TRUE)
28. │ └─tidyselect:::chr_as_locations(x, vars, call = call, arg = arg)
29. │ └─vctrs::vec_as_location(...)
30. └─vctrs (local) <fn>()
31. └─vctrs:::stop_subscript_oob(...)

32. └─vctrs:::stop_subscript(...)
    

33.   └─rlang::abort(...)
    

Execution halted

Since there is a message with batch_effect doesn't exist
following is the output of sample.tsv

sample condition batch_effect
EOL-1 cond1 batch1
EOL-2 cond2 batch1
HL601 cond1 batch1
HL602 cond2 batch1
KG-1a1 cond1 batch1
KG-1a2 cond2 batch1
MOLT41 cond1 batch1
MOLT42 cond2 batch1
OCI-AML31 cond1 batch1
OCI-AML32 cond2 batch1
THP-11 cond1 batch1
THP-12 cond2 batch1

Is there something wrong?

[dlaehnemann]

There's nothing obviously sticking out for me.

But are you sure that your sample sheet is called sample.tsv and not samples.tsv (with an s). So maybe it is not properly loading your sample sheet.

Also, a batch_effect with just one factor level (batch1) for all samples doesn't make any sense, as the model cannot do any correction with that. If you are sure you don't have any batches, so basically if you know that all samples were handled in one big batch, all at the same time, then simply leave out the specification of batch effects. But this is rarely the case...

Also, the column names condition and batch_effect are arbitrary names that you can change, you then just need to update your model specification in the config/config.yaml file accordingly.

And finally, just look at input and output files of the rule that is failing and those that come before it. Maybe you can spot what and where it goes wrong.

[dlaehnemann]

I hope this is resolved for you, so I am closing it for now. If you run into further problems, please feel free to open new issues for those.