Error converting mtx to seurat when using singularity

[rhodesch]

Description of the bug

Using the most recent commit from dev branch, process > NFCORE_SCRNASEQ:SCRNASEQ:MTX_CONVERSION:MTX_TO_SEURAT, raises an error.

This appears to be due to a build issue when singularity pulls from Quay.io nf-core/seurat. Replacing the Seurat container in scrnaseq with Quay.io rhodesch/seurat, which is a copy of the latest 'SeuratDocker' on DockerHub satijalab/seurat
, the pipeline completes error-free.

The change can be seen here: rhodesch/scrnaseq

More generally, it seems to be quite problematic converting from a SeuratDocker Dockerfile to a singularity image file. It might be useful build a dedicated Seurat.sif from a Singularity definition/recipe file, and pulling the .sif directly. However, I can't find any information about how to contribute to nf-core container repos.

Command used and terminal output

# works:
nextflow run rhodesch/scrnaseq -r dev -profile test,singularity --outdir outdir

-[nf-core/scrnaseq] Pipeline completed successfully-
Completed at: 16-Aug-2023 14:57:57
Duration    : 4m 51s
CPU hours   : 0.3
Succeeded   : 14

# confirm built container works:
singularity shell quay.io-rhodesch-seurat-latest.img 
Singularity>
(nf-core) etrain103@exp-4-26:~/work/seurat-latest/work/singularity$ singularity run quay.io-rhodesch-seurat-latest.img 
R version 4.2.0 (2022-04-22) -- "Vigorous Calisthenics"
> library(Seurat)
Attaching SeuratObject
> 

# fails:
nextflow run nf-core/scrnaseq -r dev -profile test,singularity --outdir outdir

-[nf-core/scrnaseq] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_SCRNASEQ:SCRNASEQ:MTX_CONVERSION:MTX_TO_SEURAT (Sample_X)'

Caused by:
  Process `NFCORE_SCRNASEQ:SCRNASEQ:MTX_CONVERSION:MTX_TO_SEURAT (Sample_X)` terminated with an error exit status (255)

Command executed:

  mtx_to_seurat.R \
      *.Solo.out/Gene*/filtered/matrix.mtx.gz \
      *.Solo.out/Gene*/filtered/barcodes.tsv.gz \
      *.Solo.out/Gene*/filtered/features.tsv.gz \
      Sample_X/Sample_X_matrix.rds \
      star

Command exit status:
  255

Command output:
  (empty)

Command error:
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
  FATAL:   stat /bin/bash: no such file or directory

Work dir:
  /home/user/work/seurat-4-3-0/work/93/ccf8964addc89c3f6d38c048e63014

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details

# confirm built container works:
singularity shell quay.io-nf-core-seurat-4.3.0.img 
WARNING: passwd file doesn't exist in container, not updating
WARNING: group file doesn't exist in container, not updating
ERROR:   /bin/sh does not exist in container

singularity run quay.io-nf-core-seurat-4.3.0.img 
WARNING: passwd file doesn't exist in container, not updating
WARNING: group file doesn't exist in container, not updating
FATAL:   "R": executable file not found in $PATH

Relevant files

nextflow.log

System information

Nextflow 23.04.1
nf-core 2.9
Hardware HPC
Executor slurm
Container engine: Singularity
OS Rocky Linux "8.8 (Green Obsidian)
Version of nf-core/scrnaseq dev

[grst]

afaik nf-core doesn't host any singularity images for custom containers and relies entirely on singularity's docker-to-singularity conversion (This is different for the containers provided by Biocontainers, where galaxyhub hosts img files).

In principle, the container we have should be alright, as it runs successfully on CI. Maybe it was just a temporary issue with the download. Have you tried cleaning your docker/singularity caches and trying again?

[rhodesch]

Ah, yep that seemed to fix it. After clearing the caches, the seurat image (quay.io/nf-core/seurat:4.3.0) did docker-to-singularity conversion correctly. Entire workflow completed successfully. Thanks for the feedback. Closing issue.