Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement first pass at simple plugins #261

Merged
merged 8 commits into from
Mar 9, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/bids_app/plugins.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Plugins

Plugins are a Snakebids feature that allow you to add arbitrary behaviour to your Snakebids app after CLI arguments are parsed but before Snakemake is invoked. For example, you might add BIDS validation of an input dataset to your app via a plugin, so your app is only run if the input dataset is valid.

A plugin is simply a function that takes a {class}`SnakeBidsApp <snakebids.app.SnakeBidsApp>` as input and returns either a modified {class}`SnakeBidsApp <snakebids.app.SnakeBidsApp>` or `None`. To add one or more plugins to your {class}`SnakeBidsApp <snakebids.app.SnakeBidsApp>`, use the method {func}`add_plugins <snakebids.app.SnakeBidsApp.add_plugins>`, which will typically be easiest in a `run.py` or similar entrypoint to your app. Your plugin will have access to CLI parameters via `SnakeBidsApp.config`, but you're responsible for making sure that your app provides the properties a plugin expects to find in the config.
pvandyken marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ bids_function/overview
bids_app/overview
bids_app/config
bids_app/workflow
bids_app/plugins
```

```{toctree}
Expand Down
45 changes: 39 additions & 6 deletions snakebids/app.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
"""Tools to generate a Snakemake-based BIDS app."""
from __future__ import annotations

import argparse
import logging
import sys
from collections.abc import Iterable
from pathlib import Path
from typing import Any, Dict, List, Optional
from typing import Any, Callable, Dict, List, Optional

import attr
import boutiques.creator as bc
Expand Down Expand Up @@ -111,6 +113,32 @@ class SnakeBidsApp:
takes_self=True,
)
args: Optional[SnakebidsArgs] = None
plugins: list[Callable[[SnakeBidsApp], None | SnakeBidsApp]] = attr.Factory(list)

def add_plugins(
self, plugins: Iterable[Callable[[SnakeBidsApp], None | SnakeBidsApp]]
):
"""Supply list of methods to be called after cli parsing

Each callable in ``plugins`` should take, as a single argument, a
reference to the ``SnakeBidsApp``, and should return either:

- Nothing, in which case any changes to the SnakeBidsApp need to
pvandyken marked this conversation as resolved.
Show resolved Hide resolved
come from mutating it.
- A ``SnakeBidsApp``, which will be used to call Snakemake. Note
pvandyken marked this conversation as resolved.
Show resolved Hide resolved
that in this case, any processing of CLI arguments and
configuration must already be handled, so it is recommended to
use ``copy.deepcopy`` to copy the original ``SnakeBidsApp``.

Plugins may perform any arbitrary side effects, including validation,
pvandyken marked this conversation as resolved.
Show resolved Hide resolved
optimization, other other enhancements to the snakebids app.

CLI parameters may be read from ``SnakeBidsApp.config``. Plugins
are responsible for documenting what properties they expect to find
in the config.
"""
# pylint: disable=no-member
self.plugins.extend(plugins)

def run_snakemake(self):
"""Run snakemake with that config.
Expand Down Expand Up @@ -187,10 +215,15 @@ def run_snakemake(self):
new_config_file = args.outputdir / self.configfile_path
self.config["root"] = ""

app = self
# pylint: disable=not-an-iterable
for plugin in self.plugins:
tkkuehn marked this conversation as resolved.
Show resolved Hide resolved
app = plugin(app) or app

# Write the config file
write_config_file(
config_file=new_config_file,
data=self.config,
data=app.config,
force_overwrite=True,
)

Expand All @@ -202,14 +235,14 @@ def run_snakemake(self):
None,
[
"--snakefile",
str(self.snakefile_path),
str(app.snakefile_path),
"--directory",
str(cwd),
"--configfile",
str(new_config_file.resolve()),
*self.config["snakemake_args"],
*self.config["targets_by_analysis_level"][
self.config["analysis_level"]
*app.config["snakemake_args"],
*app.config["targets_by_analysis_level"][
app.config["analysis_level"]
],
],
)
Expand Down
7 changes: 7 additions & 0 deletions snakebids/tests/test_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,12 +117,19 @@ def test_runs_in_correct_mode(
reset_db=True,
)

def plugin(my_app):
my_app.foo = "bar"

app.add_plugins([plugin])

try:
app.run_snakemake()
except SystemExit as e:
print("System exited prematurely")
print(e)

assert app.foo == "bar"
pvandyken marked this conversation as resolved.
Show resolved Hide resolved

# First condition: outputdir is an arbitrary path
if root not in ["app", "app/results"] or (root == "app" and tail):
cwd = outputdir.resolve()
Expand Down