Major updates to documentation

- Switch theme to furo
- Use snakebids 1.0 syntax for BidsDataset
- Update code blocks with line numbers and highlighting
- Use literalincludes almost exclusively within tutorial to make
  maintenance easier
- Update non-tutorial documentation files to 1.0 syntax
- Add more links to attributes like zip_lists and path
- Update typing annotations within source code for more consistency, and
  to take advantage of auto-documentation.
- Add instructions to download example dataset
- Various formatting fixes
- Pin documentation dependencies
- Remove unused sphinx extensions
- Update package author list
- Fix version update regexes

docs/api.rst

@@ -2,6 +2,8 @@
 API
 ================
 
+
+
 snakebids
 ---------
 
@@ -20,3 +22,22 @@ snakemake_io
 
 .. automodule:: snakebids.utils.snakemake_io
     :members:
+
+
+cli
+---
+
+.. automodule:: snakebids.cli
+    :members:
+
+exceptions
+----------
+
+.. automodule:: snakebids.exceptions
+    :members:
+
+types
+-----
+
+.. automodule:: snakebids.types
+    :members:

docs/bids_app/config.md

@@ -1,3 +1,4 @@
+{#bids-app-config}
 Configuration
 =============
 
@@ -10,9 +11,9 @@ Config Variables
 
 A dictionary that describes each type of input you want to grab from an input BIDS dataset. Snakebids will parse your dataset with {func}`generate_inputs() <snakebids.generate_inputs>`, converting each input type into a {class}`BidsComponent <snakebids.BidsComponent>`. The value of each item should be a dictionary with keys ``filters`` and ``wildcards``.
 
-The value of ``filters`` should be a dictionary where each key corresponds to a BIDS entity, and the value specifies which values of that entity should be grabbed. The dictionary for each input is sent to the [PyBIDS' get() function ](pybids:bids.layout.BIDSLayout). `filters` can be set according to a few different formats:
+The value of ``filters`` should be a dictionary where each key corresponds to a BIDS entity, and the value specifies which values of that entity should be grabbed. The dictionary for each input is sent to the [PyBIDS' get() function ](#bids.layout.BIDSLayout). `filters` can be set according to a few different formats:
 
-* [string](python:str): specifies an exact value for the entity. In the following example:
+* [string](#str): specifies an exact value for the entity. In the following example:
   ```yaml
   pybids_inputs:
     bold:
@@ -21,11 +22,14 @@ The value of ``filters`` should be a dictionary where each key corresponds to a
         extension: '.nii.gz'
         datatype: 'func'
   ```
+
   the bold component would match any paths under the `func/` datatype folder, with the suffix `bold` and the extension `.nii.gz`.
+
   ```
   sub-xxx/.../func/ent1-xxx_ent2-xxx_..._bold.nii.gz
   ```
-* [boolean](python:bool): constrains presence or absence of the entity without restricting its value. `False` requires that the entity be **absent**, while `True` requires that the  entity be **present**, regardless of value.
+
+* [boolean](#bool): constrains presence or absence of the entity without restricting its value. `False` requires that the entity be **absent**, while `True` requires that the  entity be **present**, regardless of value.
   ```yaml
   pybids_inputs:
     derivs:
@@ -38,7 +42,7 @@ The value of ``filters`` should be a dictionary where each key corresponds to a
 
 In addition, the special filter `regex_search` can be set to `true`, which causes all other filters in the component to use regex matching instead of exact matching.
 
-The value of ``wildcards`` should be a list of BIDS entities. Snakebids collects the values of any entities specified and saves them in the {attr}`entities <snakebids.BidsComponent.entities>` and `zip_lists` entries of the corresponding {class}`BidsComponent <snakebids.BidsComponent>`. In other words, these are the entities to be preserved in output paths derived from the input being described. Placing an entity in `wildcards` does not require the entity be present. If an entity is not found, it will be left out of {attr}`entities <snakebids.BidsComponent.entities>`. To require the presence of an entity, place it under `filters` set to `true`.
+The value of ``wildcards`` should be a list of BIDS entities. Snakebids collects the values of any entities specified and saves them in the {attr}`entities <snakebids.BidsComponent.entities>` and {attr}`~snakebids.BidsComponent.zip_lists` entries of the corresponding {class}`BidsComponent <snakebids.BidsComponent>`. In other words, these are the entities to be preserved in output paths derived from the input being described. Placing an entity in `wildcards` does not require the entity be present. If an entity is not found, it will be left out of {attr}`entities <snakebids.BidsComponent.entities>`. To require the presence of an entity, place it under `filters` set to `true`.
 
 In the following (YAML-formatted) example, the ``bold`` input type is specified. BIDS files with the datatype ``func``, suffix ``bold``, and extension ``.nii.gz`` will be grabbed, and the ``subject``, ``session``, ``acquisition``, ``task``, and ``run`` entities of those files will be left as wildcards. The `task` entity must be present, but there must not be any `desc`.
 
@@ -61,24 +65,23 @@ pybids_inputs:
 
 ### `pybids_db_dir`
 
-PyBIDS allows for the use of a cached layout to be used in order to reduce the time required to index a BIDS dataset. A path (if provided) to save the ``PyBIDS`` layout. If ``None`` or ``''`` is provided, the layout is not saved / used. The path provided must be absolute, otherwise the database will not be used. Note, this is a variable used for an opt-in feature and must first be uncommented in the ``snakebids.yml`` file.
+PyBIDS allows for the use of a cached layout to be used in order to reduce the time required to index a BIDS dataset. A path (if provided) to save the *pybids* [layout](#bids.layout.BIDSLayout). If `None` or `''` is provided, the layout is not saved or used. The path provided must be absolute, otherwise the database will not be used.
 
 ### `pybids_db_reset`
-A boolean determining whether the existing layout should be be updated. Default behaviour does not update the existing database if one is used. Note, this is a variable used for an opt-in feature and must first be uncommented in the ``snakebids.yml`` file.
+
+A boolean determining whether the existing layout should be be updated. Default behaviour does not update the existing database if one is used.
 
 ### `analysis_levels`
 
 A list of analysis levels in the BIDS app. Typically, this will include participant and/or group. Note that the default (YAML) configuration file expects this mapping to be identified with the anchor ``analysis_levels`` to be aliased by ``parse_args``.
 
-
 ### `targets_by_analysis_level`
 
 A mapping from the name of each ``analysis_level`` to the list of rules or files to be run for that analysis level.
 
-
 ### `parse_args`
 
-A dictionary of command-line parameters to make available as part of the BIDS app. Each item of the mapping is passed to [argparse's add_argument function](https://docs.python.org/3/library/argparse.html#the-add-argument-method). A number of default entries are present in a new snakebids project's config file that structure the BIDS app's CLI, but additional command-line arguments can be added as necessary.
+A dictionary of command-line parameters to make available as part of the BIDS app. Each item of the mapping is passed to [argparse's add_argument function](#argparse.ArgumentParser.add_argument). A number of default entries are present in a new snakebids project's config file that structure the BIDS app's CLI, but additional command-line arguments can be added as necessary.
 
 
 ### `debug`

docs/bids_app/workflow.md

@@ -0,0 +1,57 @@
+Workflows
+=========
+
+Snakebids workflows are constructed the same way as any other [Snakemake workflows](inv:snakemake#snakefiles/rules), but with a few additions that make it easier to work with BIDS datasets.
+
+To get access to these additions, the base Snakefile for a snakebids workflow should begin with the following boilerplate:
+
+```python
+import snakebids
+from snakebids import bids
+
+configfile: 'config/snakebids.yml'
+
+# Get input wildcards
+inputs = snakebids.generate_inputs(
+    bids_dir=config["bids_dir"],
+    pybids_inputs=config["pybids_inputs"],
+    pybids_database_dir=config.get("pybids_db_dir"),
+    pybids_reset_database=config.get("pybids_db_reset"),
+    derivatives=config.get("derivatives"),
+    participant_label=config.get("participant_label"),
+    exclude_participant_label=config.get("exclude_participant_label"),
+    use_bids_inputs=True,
+)
+
+```
+
+Snakebids workflow features
+---------------------------
+
+The [](#snakebids.bids) function generates a properly-formatted BIDS filename with the specified entities, as documented in more detail elsewhere in this documentation.
+
+[](#snakebids.generate_inputs) returns an instance of [](#snakebids.BidsDataset), a special <inv:*:*:class#dict> with keys mapping to the {class}`BidsComponents <snakebids.BidsComponent>` defined in [the config file](/bids_app/config). Each {class}`~snakebids.BidsComponent` contains a number of attributes to assist processing a BIDS dataset with snakemake. {func}`~snakebids.generate_inputs` should be called at the beginning of the workflow and assigned to a variable called `inputs`.
+
+The {attr}`~snakebids.BidsComponent.path` member of {class}`~snakebids.BidsComponent` is generated by snakebids and contains a list of matched files for every input type. Often, the first rule to be invoked will use one or more entries in `inputs.path` as the input file specification.
+
+The {attr}`~snakebids.BidsComponent.zip_lists` member of {class}`~snakebids.BidsComponent` is used with [`bids()`](#snakebids.bids) and [`expand()`](#snakefiles_expand) to fill the wildcards with corresponding values from input files. The usage pattern is as follows:
+
+```py
+expand(
+    bids(
+        root="results",
+        datatype="func",
+        suffix="bold.nii.gz",
+        **inputs.wildcards["bold"]
+    ),
+    zip,
+    **inputs.zip_lists["bold"]
+)
+```
+
+The {attr}`~snakebids.BidsComponent.wildcards` member of {class}`~snakebids.BidsComponent` is generated by snakebids and contains a dictionary mapping the wildcards for each input type to snakemake-formatted wildcards, for convenient use in the ``bids`` function.
+
+
+## Accessing the underlying *pybids* dataset
+
+In addition to mapping all of the {class}`BidsComponents <snakebids.BidsComponent>` to their names, {class}`~snakebids.BidsDataset` also has a {attr}`~snakebids.BidsDataset.layout` member which gives access to the underlying {class}`BIDSLayout <bids.layout.BIDSLayout>`. This can be used to access advanced pybids features not covered by `snakebids`. Note that if `custom_paths` are specified for every {class}`BidsComponent <snakebids.BidsComponent>`, pybids indexing will be skipped and {attr}`~snakebids.BidsDataset.layout` will be set to `None`. If your workflow relies on accessing this {attr}`~snakebids.BidsDataset.layout`, you must ensure your users do not provide a `custom_path` for every single component, either in the config file or [via the CLI](/running_snakebids/overview) (``--path_{component}``).

docs/conf.py

@@ -9,6 +9,7 @@
 # -- Path setup --------------------------------------------------------------
 
 import datetime
+import importlib.metadata as ilm
 
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
@@ -21,7 +22,8 @@
 
 # -- Project information -----------------------------------------------------
 
-project = "snakebids"
+project = "Snakebids"
+release = ilm.version("snakebids")
 copyright = f"{datetime.date.today().year}, Ali R. Khan"
 author = "Ali R. Khan"
 
@@ -32,19 +34,36 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    "sphinx_rtd_theme",
     "sphinxarg.ext",
     "sphinx.ext.intersphinx",
     "sphinx.ext.napoleon",
     "sphinx.ext.autodoc",
     "sphinxcontrib.asciinema",
     "myst_parser",
+    "sphinx_copybutton",
 ]
 
 
+myst_enable_extensions = [
+    "attrs_block",
+]
+myst_enable_extensions = [
+    "attrs_block",
+    "attrs_inline",
+    "tasklist",
+    "deflist",
+    "fieldlist",
+]
+myst_number_code_blocks = ["python", "yaml"]
+
+
 napoleon_google_docstring = False
 napoleon_numpy_docstring = True
 
+autodoc_member_order = "bysource"
+autodoc_typehints = "description"
+autosummary_imported_members = True
+
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
@@ -60,6 +79,7 @@
 intersphinx_mapping = {
     "pybids": ("https://bids-standard.github.io/pybids/", None),
     "python": ("https://docs.python.org/3", None),
+    "snakemake": ("https://snakemake.readthedocs.io/en/stable/", None),
 }
 
 
@@ -68,9 +88,16 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = "sphinx_rtd_theme"
+html_theme = "furo"
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ["_static"]
+# templates_path = ["_templates"]
+
+sphinxcontrib_asciinema_defaults = {
+    "preload": 1,
+    "rows": 24,
+    "speed": 3,
+}

docs/requirements.txt

@@ -1,5 +1,5 @@
-docutils<0.18
-sphinx-argparse
-sphinx_rtd_theme
-sphinxcontrib-asciinema
-myst-parser
+sphinx-argparse>=0.4.0,<0.5
+sphinxcontrib-asciinema>=0.3.7,<0.4
+myst-parser>=1.0.0,<2.0
+furo>=2023.3.23,<2024
+sphinx_copybutton>=0.5.1,<0.6

docs/running_snakebids/overview.md

@@ -1,5 +1,3 @@
-(running overview)=
-
 Overview
 ========
 

docs/tutorial/step0/Snakefile

@@ -1,11 +1,9 @@
-
 rule smooth:
-    input: 
-        '../bids/sub-001/func/sub-001_task-rest_run-1_bold.nii.gz'
+    input:
+        'data/sub-001/func/sub-001_task-rest_run-1_bold.nii.gz'
     params:
         sigma = '2.12'
-    output: 
+    output:
         'results/sub-001/func/sub-001_task-rest_run-1_fwhm-5mm_bold.nii.gz'
-    shell: 
+    shell:
         'fslmaths {input} -s {params.sigma} {output}'
-

docs/tutorial/step1/Snakefile

@@ -1,10 +1,9 @@
-
 rule smooth:
-    input: 
-        '../bids/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'
+    input:
+        'data/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'
     params:
         sigma = '2.12'
-    output: 
+    output:
         'results/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_fwhm-5mm_bold.nii.gz'
-    shell: 'fslmaths {input} -s {params.sigma} {output}'
-
+    shell:
+        'fslmaths {input} -s {params.sigma} {output}'

docs/tutorial/step2/Snakefile

@@ -1,14 +1,12 @@
-
 def calc_sigma_from_fwhm(wildcards):
     return f'{float(wildcards.fwhm)/2.355:0.2f}'
 
 rule smooth:
-    input: 
-        '../bids/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'
-    params: 
+    input:
+        'data/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'
+    params:
         sigma = calc_sigma_from_fwhm
-    output: 
+    output:
         'results/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_fwhm-{fwhm}mm_bold.nii.gz'
-    shell: 
+    shell:
         'fslmaths {input} -s {params.sigma} {output}'
-

docs/tutorial/step3/Snakefile

@@ -14,7 +14,7 @@ def calc_sigma_from_fwhm(wildcards):
 
 rule smooth:
     input:
-        '../bids/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'
+        'data/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'
     params:
         sigma = calc_sigma_from_fwhm,
     output:

docs/tutorial/step4/config.yml

@@ -15,4 +15,4 @@ fwhm:
   - 20
 
 
-in_bold: '../bids/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'
+in_bold: 'data/sub-{subject}/func/sub-{subject}_task-{task}_run-{run}_bold.nii.gz'

docs/tutorial/step5/Snakefile

@@ -29,11 +29,13 @@ rule smooth:
     params:
         sigma = calc_sigma_from_fwhm
     output:
-        bids(root='results',
-                subject='{subject}',
-                task='{task}',
-                run='{run}',
-                fwhm='{fwhm}',
-                suffix='bold.nii.gz')
+        bids(
+            root='results',
+            subject='{subject}',
+            task='{task}',
+            run='{run}',
+            fwhm='{fwhm}',
+            suffix='bold.nii.gz',
+        )
     shell:
         'fslmaths {input} -s {params.sigma} {output}'