Changelog
development version¶
- Run spooker on workflow completion.
- Print the recommended citation in bibtex format with
crispin --citation. (#32, @kelly-sovacool) - Fix citation information (#38, @kelly-sovacool)
CRISPIN 0.1.1¶
Bug fixes¶
- All scripts in the bin directory are now made executable for all users on biowulf (#25).
- Data directories are copied recursively during python package installation (#26).
CRISPIN 0.1.0¶
This is the first release of CRISPIN 🎉
New features¶
- Trim adapters with cutadapt.
- Run mageck count. (#9)
mageck countonly runs if a count table isn't given.- Optional: run mageck test and mle. (#9)
mageck mleonly runs if a design matrix is given.- Optional: run drugZ. (#10)
- Optional: run BAGEL2. (#11)
- fc (fold change)
- bf (bayes factor)
- pr (precision recall)
- CLI (#16)
crispin initto initialize a project directory.crispin run--mode slurmto submit a slurm job to biowulf or frce.--main path/to/main.nfto select a different local install of crispin, or specify the repo (CCBR/CRISPIN) to get it from GitHub.
Last update: 2024-05-03
Contributing to CRISPIN¶
TODO -- describe gitflow, require PRs...
Use pre-commit hooks¶
Pre-commit can automatically format your code, check for spelling errors, etc. every time you commit.
Install pre-commit if you haven't already, then run pre-commit install to install the hooks specified in .pre-commit-config.yaml. Pre-commit will run the hooks every time you commit.
Versions¶
Increment the version number following semantic versioning1 in the VERSION file.
Changelog¶
Keep the changelog up to date with all notable changes in CHANGELOG.md2.
VS code extensions¶
If you use VS code, installing nf-core extension pack is recommended.
Installation¶
For testing and debugging, We recommend installing the dev version of crispin to a user-specific location.
git clone https://github.com/CCBR/CRISPIN
+mkdir -p ~/bin/crispin
+pip install ./CRISPIN -t ~/bin/crispin
+export PATH="$HOME/bin/crispin/bin:$PATH"
+-
semantic versioning guidelines https://semver.org/ ↩
-
changelog guidelines: https://keepachangelog.com/en/1.1.0/ ↩
Last update: 2024-05-03
Getting Started with CRISPIN¶
TODO intro paragraph
Installation¶
CRISPIN is installed on the Biowulf and FRCE clusters as part of the ccbrpipeliner module. If you'd like to run the pipeline in a different execution environment, take a look at how to run the nextflow pipeline directly.
Prepare a sample sheet¶
TODO
Initialize¶
Copy the configuration files to your current working directory
crispin init
+Run¶
TODO preview, stub, mode=slurm
TODO required params
Run preview to view processes that will run:
crispin run -profile test -preview
+Launch a stub run to view processes that will run and download containers:
crispin run -profile test,singularity -stub
+Run the test dataset using the test profile:
crispin run -profile test,singularity
+or explicitly specify the output directory and input:
crispin run -profile singularity --outdir results/test --input assets/samplesheet_test.csv
+Custom reference genome¶
TODO different required params
Create and use a custom reference genome:
crispin run -profile test -entry MAKE_REFERENCE
+crispin run -profile test -c results/test/genome/custom_genome.config
+ Last update: 2024-05-03
CRISPIN 🍪¶
**C**rispr sc**R**een seq**U**encIng analy**S**is pip**E**line
🚧 This project is under active development. It is not yet ready for production use. 🚧
Set up¶
Cruise is installed on the Biowulf HPC. For installation in other execution environments, refer to the docs.
Biowulf¶
Cruise is available on Biowulf in the ccbrpipeliner module. You'll first need to start an interactive session and create a directory from where you'll run crispin.
# start an interactive node
+sinteractive --mem=2g --cpus-per-task=2 --gres=lscratch:200
+# make a working directory for your project and go to it
+mkdir -p /data/$USER/crisprseq
+cd /data/$USER/crisprseq
+# load the ccbrpipeliener module
+module load ccbrpipeliner
+Usage¶
Initialize and run crispin with test data:
# copy the crispin config files to your current directory
+crispin init
+# preview the crispin jobs that will run with the test dataset
+crispin run --mode local -profile test -preview
+# launch a crispin run on slurm with the test dataset
+crispin run --mode slurm -profile test,biowulf
+To run crispin on your own data, you'll need to create a sample sheet. Take a look at the example: assets/samplesheet_test_biowulf.csv.
You'll also need to select an appropriate library for your dataset. CRISPIN is bundled with several libraries in assets/lib, or you can download your own. Once you've created a samplesheet with paths to your fastq files, run crispin with the --input option to specify the path to your sample sheet and --library for the path to your library file:
crispin run --mode slurm -profile biowulf --input samplesheet.csv --library assets/lib/yusa_library.csv
+Help & Contributing¶
Come across a bug? Open an issue and include a minimal reproducible example.
Have a question? Ask it in discussions.
Want to contribute to this project? Check out the contributing guidelines.
References¶
This repo was originally generated from the CCBR Nextflow Template. The template takes inspiration from nektool1 and the nf-core template. If you plan to contribute your pipeline to nf-core, don't use this template -- instead follow nf-core's instructions2.
Information on who the pipeline was developed for, and a statement if it's only been tested on Biowulf. For example:
It has been developed and tested solely on NIH HPC Biowulf.
Also include a workflow image to summarize the pipeline.
-
instructions for nf-core pipelines https://nf-co.re/docs/contributing/tutorials/creating_with_nf_core ↩
Last update: 2024-05-03
nextflow pipeline¶
You can run the nextflow pipeline directly by specifying this GitHub repo. You will need nextflow and either singularity or docker installed. In this case you don't need to run crispin init first, as the config files will be accessed directly from the GitHub repo.
nextflow run CCBR/CRISPIN -profile test,singularity
+You can specify a specific version, tag, or branch with -r:
nextflow run CCBR/CRISPIN -r v1.0.0 -profile test,singularity
+Create and use a custom reference genome:
nextflow run CCBR/CRISPIN -profile test -entry MAKE_REFERENCE
+nextflow run CCBR/CRISPIN -profile test -c results/test/genome/custom_genome.config
+ Last update: 2024-05-03
Release Guide¶
How to test a pre-release on biowulf¶
Install the development version of crispin.
# activate the conda env for development
+. "/data/CCBR_Pipeliner/db/PipeDB/Conda/etc/profile.d/conda.sh"
+conda activate py311
+
+# go to the source on biowulf and update
+cd /data/CCBR_Pipeliner/Pipelines/CRISPIN/crispin-dev
+git pull
+# optionally switch to different branch if needed
+
+# install the version to a hidden path (e.g. .v0.1.0, .v1.0.0.9000) in /data/CCBR_Pipeliner/Pipelines/CRISPIN
+cd ..
+pip install ./crispin-dev -t ./.v0.1.0
+# add it to your PATH and PYTHONPATH with:
+export PATH="$PATH:/data/CCBR_Pipeliner/Pipelines/CRISPIN/.v0.1.0/bin/"
+export PYTHONPATH="$PYTHONPATH:/data/CCBR_Pipeliner/Pipelines/CRISPIN/.v0.1.0/"
+Add CRISPIN to the ccbrpipeliner module on biowulf¶
Create a lua file in /data/CCBR_Pipeliner/modules/ccbrpipeliner, e.g. dev.lua. You can copy the most recent lua file to use as a base, then modify the following line:
source_sh("bash", "/data/CCBR_Pipeliner/Pipelines/CRISPIN/crispin-dev/bin/install.sh .v0.1.0")
+Change the version of CRISPIN as needed. Notice the dot (.) in front of the version number to indicate a hidden path.
Then use and load the module (substitute dev for whatever you named the new lua file) with:
module use /data/CCBR_Pipeliner/modules
+module load ccbrpipeliner/dev
+ Last update: 2024-05-03
**C**rispr sc**R**een seq**U**encIng analy**S**is pip**E**line
\ud83d\udea7 This project is under active development. It is not yet ready for production use. \ud83d\udea7
"},{"location":"#set-up","title":"Set up","text":"Cruise is installed on the Biowulf HPC. For installation in other execution environments, refer to the docs.
"},{"location":"#biowulf","title":"Biowulf","text":"Cruise is available on Biowulf in the ccbrpipeliner module. You'll first need to start an interactive session and create a directory from where you'll run crispin.
# start an interactive node\nsinteractive --mem=2g --cpus-per-task=2 --gres=lscratch:200\n# make a working directory for your project and go to it\nmkdir -p /data/$USER/crisprseq\ncd /data/$USER/crisprseq\n# load the ccbrpipeliener module\nmodule load ccbrpipeliner\nInitialize and run crispin with test data:
# copy the crispin config files to your current directory\ncrispin init\n# preview the crispin jobs that will run with the test dataset\ncrispin run --mode local -profile test -preview\n# launch a crispin run on slurm with the test dataset\ncrispin run --mode slurm -profile test,biowulf\nTo run crispin on your own data, you'll need to create a sample sheet. Take a look at the example: assets/samplesheet_test_biowulf.csv.
You'll also need to select an appropriate library for your dataset. CRISPIN is bundled with several libraries in assets/lib, or you can download your own. Once you've created a samplesheet with paths to your fastq files, run crispin with the --input option to specify the path to your sample sheet and --library for the path to your library file:
crispin run --mode slurm -profile biowulf --input samplesheet.csv --library assets/lib/yusa_library.csv\nCome across a bug? Open an issue and include a minimal reproducible example.
Have a question? Ask it in discussions.
Want to contribute to this project? Check out the contributing guidelines.
"},{"location":"#references","title":"References","text":"This repo was originally generated from the CCBR Nextflow Template. The template takes inspiration from nektool1 and the nf-core template. If you plan to contribute your pipeline to nf-core, don't use this template -- instead follow nf-core's instructions2.
Information on who the pipeline was developed for, and a statement if it's only been tested on Biowulf. For example:
It has been developed and tested solely on NIH HPC Biowulf.
Also include a workflow image to summarize the pipeline.
-
nektool https://github.com/beardymcjohnface/nektool \u21a9
-
instructions for nf-core pipelines https://nf-co.re/docs/contributing/tutorials/creating_with_nf_core \u21a9
- Run spooker on workflow completion.
- Print the recommended citation in bibtex format with
crispin --citation. (#32, @kelly-sovacool) - Fix citation information (#38, @kelly-sovacool)
- All scripts in the bin directory are now made executable for all users on biowulf (#25).
- Data directories are copied recursively during python package installation (#26).
This is the first release of CRISPIN \ud83c\udf89
"},{"location":"changelog/#new-features","title":"New features","text":"- Trim adapters with cutadapt.
- Run mageck count. (#9)
mageck countonly runs if a count table isn't given.- Optional: run mageck test and mle. (#9)
mageck mleonly runs if a design matrix is given.- Optional: run drugZ. (#10)
- Optional: run BAGEL2. (#11)
- fc (fold change)
- bf (bayes factor)
- pr (precision recall)
- CLI (#16)
crispin initto initialize a project directory.crispin run--mode slurmto submit a slurm job to biowulf or frce.--main path/to/main.nfto select a different local install of crispin, or specify the repo (CCBR/CRISPIN) to get it from GitHub.
TODO -- describe gitflow, require PRs...
"},{"location":"contributing/#use-pre-commit-hooks","title":"Use pre-commit hooks","text":"Pre-commit can automatically format your code, check for spelling errors, etc. every time you commit.
Install pre-commit if you haven't already, then run pre-commit install to install the hooks specified in .pre-commit-config.yaml. Pre-commit will run the hooks every time you commit.
Increment the version number following semantic versioning1 in the VERSION file.
Keep the changelog up to date with all notable changes in CHANGELOG.md2.
If you use VS code, installing nf-core extension pack is recommended.
"},{"location":"contributing/#installation","title":"Installation","text":"For testing and debugging, We recommend installing the dev version of crispin to a user-specific location.
git clone https://github.com/CCBR/CRISPIN\nmkdir -p ~/bin/crispin\npip install ./CRISPIN -t ~/bin/crispin\nexport PATH=\"$HOME/bin/crispin/bin:$PATH\"\n-
semantic versioning guidelines https://semver.org/ \u21a9
-
changelog guidelines: https://keepachangelog.com/en/1.1.0/ \u21a9
TODO intro paragraph
"},{"location":"getting-started/#installation","title":"Installation","text":"CRISPIN is installed on the Biowulf and FRCE clusters as part of the ccbrpipeliner module. If you'd like to run the pipeline in a different execution environment, take a look at how to run the nextflow pipeline directly.
TODO
"},{"location":"getting-started/#initialize","title":"Initialize","text":"Copy the configuration files to your current working directory
crispin init\nTODO preview, stub, mode=slurm
TODO required params
Run preview to view processes that will run:
crispin run -profile test -preview\nLaunch a stub run to view processes that will run and download containers:
crispin run -profile test,singularity -stub\nRun the test dataset using the test profile:
crispin run -profile test,singularity\nor explicitly specify the output directory and input:
crispin run -profile singularity --outdir results/test --input assets/samplesheet_test.csv\nTODO different required params
Create and use a custom reference genome:
crispin run -profile test -entry MAKE_REFERENCE\ncrispin run -profile test -c results/test/genome/custom_genome.config\nYou can run the nextflow pipeline directly by specifying this GitHub repo. You will need nextflow and either singularity or docker installed. In this case you don't need to run crispin init first, as the config files will be accessed directly from the GitHub repo.
nextflow run CCBR/CRISPIN -profile test,singularity\nYou can specify a specific version, tag, or branch with -r:
nextflow run CCBR/CRISPIN -r v1.0.0 -profile test,singularity\nCreate and use a custom reference genome:
nextflow run CCBR/CRISPIN -profile test -entry MAKE_REFERENCE\nnextflow run CCBR/CRISPIN -profile test -c results/test/genome/custom_genome.config\nInstall the development version of crispin.
# activate the conda env for development\n. \"/data/CCBR_Pipeliner/db/PipeDB/Conda/etc/profile.d/conda.sh\"\nconda activate py311\n\n# go to the source on biowulf and update\ncd /data/CCBR_Pipeliner/Pipelines/CRISPIN/crispin-dev\ngit pull\n# optionally switch to different branch if needed\n\n# install the version to a hidden path (e.g. .v0.1.0, .v1.0.0.9000) in /data/CCBR_Pipeliner/Pipelines/CRISPIN\ncd ..\npip install ./crispin-dev -t ./.v0.1.0\n# add it to your PATH and PYTHONPATH with:\nexport PATH=\"$PATH:/data/CCBR_Pipeliner/Pipelines/CRISPIN/.v0.1.0/bin/\"\nexport PYTHONPATH=\"$PYTHONPATH:/data/CCBR_Pipeliner/Pipelines/CRISPIN/.v0.1.0/\"\nCreate a lua file in /data/CCBR_Pipeliner/modules/ccbrpipeliner, e.g. dev.lua. You can copy the most recent lua file to use as a base, then modify the following line:
source_sh(\"bash\", \"/data/CCBR_Pipeliner/Pipelines/CRISPIN/crispin-dev/bin/install.sh .v0.1.0\")\nChange the version of CRISPIN as needed. Notice the dot (.) in front of the version number to indicate a hidden path.
Then use and load the module (substitute dev for whatever you named the new lua file) with:
module use /data/CCBR_Pipeliner/modules\nmodule load ccbrpipeliner/dev\n