Changelog
CRUISE development version¶
New features¶
- Trim adapters with cutadapt.
- Run mageck count. (#9)
mageck countonly runs if a count table isn't given.- Optional: run mageck test and mle. (#9)
mageck mleonly runs if a design matrix is given.- Optional: run drugZ. (#10)
- Optional: run BAGEL2. (#11)
- fc (fold change)
- bf (bayes factor)
- pr (precision recall)
- CLI (#16)
cruise initto initialize a project directory.cruise run--mode slurmto submit a slurm job to biowulf or frce.--main path/to/main.nfto select a different local install of cruise, or specify the repo (CCBR/CRUISE) to get it from GitHub.
Last update: 2023-10-16
Contributing to CRUISE¶
TODO -- describe gitflow, require PRs...
Use pre-commit hooks¶
Pre-commit can automatically format your code, check for spelling errors, etc. every time you commit.
Install pre-commit if you haven't already, then run pre-commit install to install the hooks specified in .pre-commit-config.yaml. Pre-commit will run the hooks every time you commit.
Versions¶
Increment the version number following semantic versioning1 in the VERSION file.
Changelog¶
Keep the changelog up to date with all notable changes in CHANGELOG.md2.
VS code extensions¶
If you use VS code, installing nf-core extension pack is recommended.
Installation¶
For testing and debugging, We recommend installing the dev version of cruise to a user-specific location.
git clone https://github.com/CCBR/CRUISE
+mkdir -p ~/bin/cruise
+pip install ./CRUISE -t ~/bin/cruise
+export PATH="$HOME/bin/cruise/bin:$PATH"
+-
semantic versioning guidelines https://semver.org/ ↩
-
changelog guidelines: https://keepachangelog.com/en/1.1.0/ ↩
Last update: 2023-10-16
Getting Started with CRUISE¶
TODO intro paragraph
Installation¶
CRUISE is installed on the Biowulf and FRCE clusters as part of the ccbrpipeliner module. If you'd like to run the pipeline in a different execution environment, take a look at how to run the nextflow pipeline directly.
Prepare a sample sheet¶
TODO
Initialize¶
Copy the configuration files to your current working directory
cruise init
+Run¶
TODO preview, stub, mode=slurm
TODO required params
Run preview to view processes that will run:
cruise run -profile test -preview
+Launch a stub run to view processes that will run and download containers:
cruise run -profile test,singularity -stub
+Run the test dataset using the test profile:
cruise run -profile test,singularity
+or explicitly specify the output directory and input:
cruise run -profile singularity --outdir results/test --input assets/samplesheet_test.csv
+Custom reference genome¶
TODO different required params
Create and use a custom reference genome:
cruise run -profile test -entry MAKE_REFERENCE
+cruise run -profile test -c results/test/genome/custom_genome.config
+ Last update: 2023-10-16
CRUISE 🛳️¶
**C**rispr sc**R**een seq**U**encIng analy**S**is pip**E**line
🚧 This project is under active development. It is not yet ready for production use. 🚧
Set up¶
Cruise is installed on the Biowulf HPC. For installation in other execution environments, refer to the docs.
Biowulf¶
Cruise is available on Biowulf in the ccbrpipeliner module. You'll first need to start an interactive session and create a directory from where you'll run cruise.
# start an interactive node
+sinteractive --mem=2g --cpus-per-task=2 --gres=lscratch:200
+# make a working directory for your project and go to it
+mkdir -p /data/$USER/chipseq
+cd /data/$USER/chipseq
+# load the ccbrpipeliener module
+module load ccbrpipeliner
+Usage¶
Initialize and run cruise with test data:
# copy the cruise config files to your current directory
+cruise init
+# preview the cruise jobs that will run with the test dataset
+cruise run --mode local -profile test -preview
+# launch a cruise run on slurm with the test dataset
+cruise run --mode slurm -profile test,biowulf
+To run cruise on your own data, you'll need to create a sample sheet. Take a look at the example: assets/samplesheet_test.csv.
Once you've created a samplesheet with paths to your fastq files, run cruise with the --input option to specify the path to your sample sheet:
cruise run --mode slurm -profile biowulf --input samplesheet.csv
+Help & Contributing¶
Come across a bug? Open an issue and include a minimal reproducible example.
Have a question? Ask it in discussions.
Want to contribute to this project? Check out the contributing guidelines.
References¶
This repo was originally generated from the CCBR Nextflow Template. The template takes inspiration from nektool1 and the nf-core template. If you plan to contribute your pipeline to nf-core, don't use this template -- instead follow nf-core's instructions2.
Information on who the pipeline was developed for, and a statement if it's only been tested on Biowulf. For example:
It has been developed and tested solely on NIH HPC Biowulf.
Also include a workflow image to summarize the pipeline.
-
instructions for nf-core pipelines https://nf-co.re/docs/contributing/tutorials/creating_with_nf_core ↩
Last update: 2023-10-16
nextflow pipeline¶
You can run the nextflow pipeline directly by specifying this GitHub repo. You will need nextflow and either singularity or docker installed. In this case you don't need to run cruise init first, as the config files will be accessed directly from the GitHub repo.
nextflow run CCBR/CRUISE -profile test,singularity
+You can specify a specific version, tag, or branch with -r:
nextflow run CCBR/CRUISE -r v1.0.0 -profile test,singularity
+Create and use a custom reference genome:
nextflow run CCBR/CRUISE -profile test -entry MAKE_REFERENCE
+nextflow run CCBR/CRUISE -profile test -c results/test/genome/custom_genome.config
+ Last update: 2023-10-16
Release Guide¶
How to test a pre-release on biowulf¶
Install the development version of cruise.
# activate the conda env for development
+. "/data/CCBR_Pipeliner/db/PipeDB/Conda/etc/profile.d/conda.sh"
+conda activate py311
+
+# go to the source on biowulf and update
+cd /data/CCBR_Pipeliner/Pipelines/CRUISE/cruise-dev
+git pull
+# optionally switch to different branch if needed
+
+# install the version to a hidden path (e.g. .v0.1.0, .v1.0.0.9000) in /data/CCBR_Pipeliner/Pipelines/CRUISE
+cd ..
+pip install ./cruise-dev -t ./.v0.1.0
+# add it to your PATH and PYTHONPATH with:
+export PATH="$PATH:/data/CCBR_Pipeliner/Pipelines/CRUISE/.v0.1.0/bin/"
+export PYTHONPATH="$PYTHONPATH:/data/CCBR_Pipeliner/Pipelines/CRUISE/.v0.1.0/"
+Add CRUISE to the ccbrpipeliner module on biowulf¶
Create a lua file in /data/CCBR_Pipeliner/modules/ccbrpipeliner, e.g. dev.lua. You can copy the most recent lua file to use as a base, then modify the following line:
source_sh("bash", "/data/CCBR_Pipeliner/Pipelines/CHAMPAGNE/cruise-dev/bin/install.sh .v0.1.0")
+Change the version of CRUISE as needed. Notice the dot (.) in front of the version number to indicate a hidden path.
Then use and load the module (substitute dev for whatever you named the new lua file) with:
module use /data/CCBR_Pipeliner/modules
+module load ccbrpipeliner/dev
+ Last update: 2023-10-16
**C**rispr sc**R**een seq**U**encIng analy**S**is pip**E**line
\ud83d\udea7 This project is under active development. It is not yet ready for production use. \ud83d\udea7
"},{"location":"#set-up","title":"Set up","text":"Cruise is installed on the Biowulf HPC. For installation in other execution environments, refer to the docs.
"},{"location":"#biowulf","title":"Biowulf","text":"Cruise is available on Biowulf in the ccbrpipeliner module. You'll first need to start an interactive session and create a directory from where you'll run cruise.
# start an interactive node\nsinteractive --mem=2g --cpus-per-task=2 --gres=lscratch:200\n# make a working directory for your project and go to it\nmkdir -p /data/$USER/chipseq\ncd /data/$USER/chipseq\n# load the ccbrpipeliener module\nmodule load ccbrpipeliner\nInitialize and run cruise with test data:
# copy the cruise config files to your current directory\ncruise init\n# preview the cruise jobs that will run with the test dataset\ncruise run --mode local -profile test -preview\n# launch a cruise run on slurm with the test dataset\ncruise run --mode slurm -profile test,biowulf\nTo run cruise on your own data, you'll need to create a sample sheet. Take a look at the example: assets/samplesheet_test.csv.
Once you've created a samplesheet with paths to your fastq files, run cruise with the --input option to specify the path to your sample sheet:
cruise run --mode slurm -profile biowulf --input samplesheet.csv\nCome across a bug? Open an issue and include a minimal reproducible example.
Have a question? Ask it in discussions.
Want to contribute to this project? Check out the contributing guidelines.
"},{"location":"#references","title":"References","text":"This repo was originally generated from the CCBR Nextflow Template. The template takes inspiration from nektool1 and the nf-core template. If you plan to contribute your pipeline to nf-core, don't use this template -- instead follow nf-core's instructions2.
Information on who the pipeline was developed for, and a statement if it's only been tested on Biowulf. For example:
It has been developed and tested solely on NIH HPC Biowulf.
Also include a workflow image to summarize the pipeline.
-
nektool https://github.com/beardymcjohnface/nektool \u21a9
-
instructions for nf-core pipelines https://nf-co.re/docs/contributing/tutorials/creating_with_nf_core \u21a9
- Trim adapters with cutadapt.
- Run mageck count. (#9)
mageck countonly runs if a count table isn't given.- Optional: run mageck test and mle. (#9)
mageck mleonly runs if a design matrix is given.- Optional: run drugZ. (#10)
- Optional: run BAGEL2. (#11)
- fc (fold change)
- bf (bayes factor)
- pr (precision recall)
- CLI (#16)
cruise initto initialize a project directory.cruise run--mode slurmto submit a slurm job to biowulf or frce.--main path/to/main.nfto select a different local install of cruise, or specify the repo (CCBR/CRUISE) to get it from GitHub.
TODO -- describe gitflow, require PRs...
"},{"location":"contributing/#use-pre-commit-hooks","title":"Use pre-commit hooks","text":"Pre-commit can automatically format your code, check for spelling errors, etc. every time you commit.
Install pre-commit if you haven't already, then run pre-commit install to install the hooks specified in .pre-commit-config.yaml. Pre-commit will run the hooks every time you commit.
Increment the version number following semantic versioning1 in the VERSION file.
Keep the changelog up to date with all notable changes in CHANGELOG.md2.
If you use VS code, installing nf-core extension pack is recommended.
"},{"location":"contributing/#installation","title":"Installation","text":"For testing and debugging, We recommend installing the dev version of cruise to a user-specific location.
git clone https://github.com/CCBR/CRUISE\nmkdir -p ~/bin/cruise\npip install ./CRUISE -t ~/bin/cruise\nexport PATH=\"$HOME/bin/cruise/bin:$PATH\"\n-
semantic versioning guidelines https://semver.org/ \u21a9
-
changelog guidelines: https://keepachangelog.com/en/1.1.0/ \u21a9
TODO intro paragraph
"},{"location":"getting-started/#installation","title":"Installation","text":"CRUISE is installed on the Biowulf and FRCE clusters as part of the ccbrpipeliner module. If you'd like to run the pipeline in a different execution environment, take a look at how to run the nextflow pipeline directly.
TODO
"},{"location":"getting-started/#initialize","title":"Initialize","text":"Copy the configuration files to your current working directory
cruise init\nTODO preview, stub, mode=slurm
TODO required params
Run preview to view processes that will run:
cruise run -profile test -preview\nLaunch a stub run to view processes that will run and download containers:
cruise run -profile test,singularity -stub\nRun the test dataset using the test profile:
cruise run -profile test,singularity\nor explicitly specify the output directory and input:
cruise run -profile singularity --outdir results/test --input assets/samplesheet_test.csv\nTODO different required params
Create and use a custom reference genome:
cruise run -profile test -entry MAKE_REFERENCE\ncruise run -profile test -c results/test/genome/custom_genome.config\nYou can run the nextflow pipeline directly by specifying this GitHub repo. You will need nextflow and either singularity or docker installed. In this case you don't need to run cruise init first, as the config files will be accessed directly from the GitHub repo.
nextflow run CCBR/CRUISE -profile test,singularity\nYou can specify a specific version, tag, or branch with -r:
nextflow run CCBR/CRUISE -r v1.0.0 -profile test,singularity\nCreate and use a custom reference genome:
nextflow run CCBR/CRUISE -profile test -entry MAKE_REFERENCE\nnextflow run CCBR/CRUISE -profile test -c results/test/genome/custom_genome.config\nInstall the development version of cruise.
# activate the conda env for development\n. \"/data/CCBR_Pipeliner/db/PipeDB/Conda/etc/profile.d/conda.sh\"\nconda activate py311\n\n# go to the source on biowulf and update\ncd /data/CCBR_Pipeliner/Pipelines/CRUISE/cruise-dev\ngit pull\n# optionally switch to different branch if needed\n\n# install the version to a hidden path (e.g. .v0.1.0, .v1.0.0.9000) in /data/CCBR_Pipeliner/Pipelines/CRUISE\ncd ..\npip install ./cruise-dev -t ./.v0.1.0\n# add it to your PATH and PYTHONPATH with:\nexport PATH=\"$PATH:/data/CCBR_Pipeliner/Pipelines/CRUISE/.v0.1.0/bin/\"\nexport PYTHONPATH=\"$PYTHONPATH:/data/CCBR_Pipeliner/Pipelines/CRUISE/.v0.1.0/\"\nCreate a lua file in /data/CCBR_Pipeliner/modules/ccbrpipeliner, e.g. dev.lua. You can copy the most recent lua file to use as a base, then modify the following line:
source_sh(\"bash\", \"/data/CCBR_Pipeliner/Pipelines/CHAMPAGNE/cruise-dev/bin/install.sh .v0.1.0\")\nChange the version of CRUISE as needed. Notice the dot (.) in front of the version number to indicate a hidden path.
Then use and load the module (substitute dev for whatever you named the new lua file) with:
module use /data/CCBR_Pipeliner/modules\nmodule load ccbrpipeliner/dev\n