Skip to content

Commit

Permalink
Progress
Browse files Browse the repository at this point in the history
  • Loading branch information
pvandyken committed Sep 15, 2023
1 parent e7801c0 commit 06f4961
Show file tree
Hide file tree
Showing 23 changed files with 443 additions and 282 deletions.
521 changes: 341 additions & 180 deletions poetry.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ scipy = [
{ version = ">=1.10.0,<=1.10.1", python = "<3.9" },
{ version = ">=1.10.0", python = ">=3.9" }
]
copier = "^8.1.0"

[tool.poetry.group.dev.dependencies]
black = "^23.1.0"
Expand Down
12 changes: 0 additions & 12 deletions snakebids/project_template/cookiecutter.json

This file was deleted.

45 changes: 45 additions & 0 deletions snakebids/project_template/copier.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
full_name:
type: str
help: What is your name?
placeholder: " optional"

email:
type: str
help: What is your email?

github:
type: str
help: What is your github username?

app_full_name:
type: str
help: What is the name of your app?
default: "{{ dst_path }}"
validator: '{% if not app_full_name %}Required{% endif %}'

app_description:
type: str
help: Provide a brief description of your app

app_version:
default: "0.1.0"
help: Starting version number for your app

create_doc_template:
type: bool
help: Would you to set up basic documentation?

bids_version:
default: "1.8.0"
when: false

snakebids_version:
default: "0.0.0"
when: false

name_slug:
default: "{{ app_full_name|lower|replace(' ', '_')|replace('-', '_') }}"
when: false

_jinja_extensions:
- jinja2_time.TimeExtension
37 changes: 0 additions & 37 deletions snakebids/project_template/hooks/post_gen_project.py

This file was deleted.

3 changes: 3 additions & 0 deletions snakebids/project_template/{{ name_slug }}/README.md.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# {{app_full_name}}

{{ app_description }}
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@
],
entry_points={
"console_scripts": [
"{{cookiecutter.__app_name}}={{cookiecutter.__app_name}}.run:main"
"{{name_slug}}={{name_slug}}.run:main"
]
},
install_requires=[
"snakebids>={{cookiecutter._snakebids_version}}",
"snakebids>={{snakebids_version}}",
"snakemake",
],
python_requires=">=3.7",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@
# -- Project information -----------------------------------------------------


project = "{{cookiecutter.__app_name}}"
copyright = "{% now 'utc', '%Y' %}, {{cookiecutter.full_name}}"
author = "{{cookiecutter.full_name}}"
project = "{{name_slug}}"
copyright = "{% now 'utc', '%Y' %}, {{full_name}}"
author = "{{full_name}}"


# -- General configuration ---------------------------------------------------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Install from github with pip:

```bash
pip install -e git+https://github.com/{{cookiecutter.github}}/{{cookiecutter.__app_name}}#egg={{cookiecutter.__app_name}}
pip install -e git+https://github.com/{{github}}/{{name_slug}}#egg={{name_slug}}
```

Note: you can re-run this command to re-install with the latest version
Expand All @@ -13,13 +13,13 @@ Note: you can re-run this command to re-install with the latest version
Do a dry-run first (`-n`) and simply print (`-p`) what would be run:

```bash
{{cookiecutter.__app_name}} /path/to/bids/dir /path/to/output/dir participant -np
{{name_slug}} /path/to/bids/dir /path/to/output/dir participant -np
```

Run the app, using all cores::

```bash
{{cookiecutter.__app_name}} /path/to/bids/dir /path/to/output/dir participant --cores all
{{name_slug}} /path/to/bids/dir /path/to/output/dir participant --cores all
```

If any workflow rules require containers, then run with the `--use-singularity` option.
Expand All @@ -35,22 +35,22 @@ summarize outputs, by using the `report(...)` function on any snakemake output.
To generate a report, run:

```bash
{{cookiecutter.__app_name}} /path/to/bids/dir /path/to/output/dir participant --report
{{name_slug}} /path/to/bids/dir /path/to/output/dir participant --report
```

## Compute Canada Instructions

### Setting up a dev environment

Here are some instructions to get your python environment set-up on graham to run {{ cookiecutter.__app_name }}:
Here are some instructions to get your python environment set-up on graham to run {{name_slug}}:

# Create a virtualenv and activate it:

```bash
cd $SCRATCH
module load python/3
virtualenv venv_{{ cookiecutter.__app_name }}
source venv_{{ cookiecutter.__app_name }}/bin/activate
virtualenv venv_{{name_slug}}
source venv_{{name_slug}}/bin/activate
```

# Follow the steps above to install from github repository
Expand All @@ -69,23 +69,23 @@ These are used in the instructions below.
In an interactive job (for testing):

```bash
regularInteractive -n 8 {{ cookiecutter.__app_name }} bids_dir out_dir participant --participant_label 001 -j 8
regularInteractive -n 8 {{name_slug}} bids_dir out_dir participant --participant_label 001 -j 8
```

Submitting a job (for larger cores, more subjects), still single job, but snakemake will parallelize over the 32 cores:

```bash
regularSubmit -j Fat {{ cookiecutter.__app_name }} bids_dir out_dir participant -j 32
regularSubmit -j Fat {{name_slug}} bids_dir out_dir participant -j 32
```

Scaling up to ~hundred subjects (needs cc-slurm snakemake profile installed), submits 1 16core job per subject:

```bash
{{ cookiecutter.__app_name }} bids_dir out_dir participant --profile cc-slurm
{{name_slug}} bids_dir out_dir participant --profile cc-slurm
```

Scaling up to even more subjects (uses group-components to bundle multiple subjects in each job), 1 32core job for N subjects (e.g. 10):

```bash
{{ cookiecutter.__app_name }} bids_dir out_dir participant --profile cc-slurm --group-components subj=10
```
{{name_slug}} bids_dir out_dir participant --profile cc-slurm --group-components subj=10
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
docutils<0.18
sphinx-argparse
sphinx_rtd_theme
snakebids=={{ snakebids_version }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Command line interface

```{argparse}
:filename: ../{{ name_slug }}/run.py
:func: get_parser
:prog: {{ name_slug }}
```
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
:module: snakemake
:func: get_argument_parser
:prog: snakemake
```
```
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ parse_args:
# below are examples for plugin and custom parameters (e.g. config['smoothing_fwhm'])
--skip_bids_validation:
help: 'Skip validation of BIDS dataset. BIDS validation is performed by
default using the bids-validator plugin (if installed/enabled) or with the pybids
default using the bids-validator plugin (if installed/enabled) or with the pybids
validator implementation (if bids-validator is not installed/enabled).'
dest: "plugins.validator.skip"
action: "store_true"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{%
set generated = {
"Name": name_slug,
"Version": app_version,
}
-%}
{%- if github -%}
{%- set _ = generated.update({"CodeURL": "https://github.com/" + github + "/" + name_slug}) -%}
{%- endif -%}
{%- if full_name -%}
{%- set _ = generated.update({"Author": full_name}) -%}
{%- endif -%}
{%- if email -%}
{%- set _ = generated.update({"AuthorEmail": email}) -%}
{%- endif -%}
{{- {
"Name": "Dataset generated by " + name_slug,
"BIDSVersion": bids_version,
"DatasetType": "derivative",
"GeneratedBy": [generated],
} | to_json(indent=2)
}}
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

def get_parser():
"""Exposes parser for sphinx doc generation, cwd is the docs dir"""
app = SnakeBidsApp("../{{ cookiecutter.__app_name }}")
app = SnakeBidsApp("../{{ name_slug }}")
return app.parser


Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

0 comments on commit 06f4961

Please sign in to comment.