Restructure documentation and document how to run tests locally (#879)

* Clean up organization and labeling. * Remove some more reconstruction stuff. * Update README.rst * Document how to run tests locally. * Describe how to evaluate correct behavior. * Update docs/usage.rst Co-authored-by: Matt Cieslak <[email protected]> * Update usage.rst --------- Co-authored-by: Matt Cieslak <[email protected]>
PennLINC · Nov 18, 2024 · f268f2a · f268f2a
1 parent dcb0f30
commit f268f2a
Show file tree

Hide file tree

Showing 18 changed files with 219 additions and 235 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -459,7 +459,6 @@ jobs:
               echo "the command ``git fetch --tags --verbose`` and push"
               echo "them to your fork with ``git push origin --tags``"
             fi
-            sed -i -E "s/(var version = )'[A-Za-z0-9.-]+'/\1'${CIRCLE_TAG:-$THISVERSION}'/" docs/citing.rst
             sed -i "s/title = {qsiprep}/title = {qsiprep ${CIRCLE_TAG:-$THISVERSION}}/" qsiprep/data/boilerplate.bib
             # Build docker image
             e=1 && for i in {1..5}; do

diff --git a/README.rst b/README.rst
@@ -1,7 +1,7 @@
 .. include:: links.rst
 
-QSIprep: Preprocessing and analysis of q-space images
-=======================================================
+QSIPrep: Preprocessing and analysis of q-space images
+=====================================================
 
 .. image:: https://img.shields.io/badge/Source%20Code-pennlinc%2Fqsiprep-purple
   :target: https://github.com/PennLINC/qsiprep
@@ -33,7 +33,7 @@ Full documentation at https://qsiprep.readthedocs.io
 About
 -----
 
-``qsiprep`` configures pipelines for processing diffusion-weighted MRI (dMRI) data.
+*QSIPrep* configures pipelines for processing diffusion-weighted MRI (dMRI) data.
 The main features of this software are
 
   1. A BIDS-app approach to preprocessing nearly all kinds of modern diffusion MRI data.
@@ -50,7 +50,7 @@ The main features of this software are
 .. _preprocessing_def:
 
 Preprocessing
-~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~
 
 The preprocessing pipelines are built based on the available BIDS inputs, ensuring that fieldmaps
 are handled correctly. The preprocessing workflow performs head motion correction, susceptibility
@@ -61,17 +61,21 @@ using ANTs_ and tissue segmentation.
 .. _reconstruction_def:
 
 Reconstruction
-~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~
 
 The outputs from the :ref:`preprocessing_def` pipelines can be reconstructed in many other
-software packages. We provide a curated set of :ref:`recon_workflows` in ``qsiprep``
+software packages.
+We recommend passing *QSIPrep* derivatives along to
+`QSIRecon <https://qsirecon.readthedocs.io/en/latest/>`_,
+which provides a curated set of reconstruction workflows
 that can run ODF/FOD reconstruction, tractography, Fixel estimation and regional
 connectivity.
 
 
 Note
-------
+----
 
-The ``qsiprep`` pipeline uses much of the code from ``FMRIPREP``. It is critical
-to note that the similarities in the code **do not imply that the authors of
-FMRIPREP in any way endorse or support this code or its pipelines**.
+The *QSIPrep* pipeline uses much of the code from *fMRIPrep*.
+It is critical to note that the similarities in the code
+**do not imply that the authors of QSIPrep in any way endorse or support this code or its
+pipelines**.
diff --git a/docs/api.rst b/docs/api.rst
@@ -17,7 +17,7 @@ Library API
 ***********
 
 Preprocessing Workflows
------------------------
+=======================
 
 .. toctree::
    :glob:
@@ -28,17 +28,8 @@ Preprocessing Workflows
    api/qsiprep.workflows.fieldmap
 
 
-Reconstruction Workflows
-------------------------
-
-.. toctree::
-   :glob:
-
-   api/qsiprep.workflows.recon
-
-
 Other Utilities
----------------
+===============
 
 .. toctree::
    :glob:

diff --git a/docs/changes.rst b/docs/changes.rst
@@ -1,7 +1,7 @@
 .. include:: links.rst
 
-----------
+##########
 What's new
-----------
+##########
 
 .. include:: ../CHANGES.rst
diff --git a/docs/citing.rst b/docs/citing.rst
diff --git a/docs/comparisons.md b/docs/comparisons.md
@@ -8,7 +8,6 @@ their current feature sets. These other
  * [MRtrix3_connectome](https://github.com/BIDS-Apps/MRtrix3_connectome)
  * [dMRIPrep](https://github.com/nipreps/dmriprep)
 
- dMRIPrep exclusively performs preprocessing and is therefore omitted from the [reconstruction](#reconstruction) and [tractography](#tractography) sections.
 
 ## Supported Sampling Schemes
 
@@ -43,34 +42,9 @@ their current feature sets. These other
 | HTML Report                                         |            ✔             |           ✘            |            ✔            |         ✘          |        ✔         |
 | Containerized                                       |            ✔             |           ✔            |            ✔            |         ✔          |        ✔         |
 
-
-## Reconstruction
-
-|                           | QSIPrep | Tractoflow | PreQual | MRtrix3_connectome |
-| ------------------------- | :-----: | :--------: | :-----: | :----------------: |
-| MRTrix3 MSMT CSD          |    ✔    |     ✘      |    ✘    |         ✔          |
-| CSD                       | MRtrix3 |    DIPY    |    ✘    |      MRtrix3       |
-| Single Shell 3-Tissue CSD |    ✔    |     ✘      |    ✘    |         ✘          |
-| DTI Metrics               |    ✔    |     ✔      |    ✔    |         ✔          |
-| DSI Studio GQI            |    ✔    |     ✘      |    ✘    |         ✘          |
-| MAPMRI                    |    ✔    |     ✘      |    ✘    |         ✘          |
-| 3dSHORE                   |    ✔    |     ✘      |    ✘    |         ✘          |
-
-## Tractography
-
-|                                       | QSIPrep | Tractoflow | PreQual | MRtrix3_connectome |
-| ------------------------------------- | :-----: | :--------: | :-----: | :----------------: |
-| DIPY Particle Filtering               |    ✘    |     ✔      |    ✘    |         ✘          |
-| MRtrix3 iFOD2                         |    ✔    |     ✘      |    ✘    |         ✔          |
-| Anatomically constrained Tractography |    ✔    |     ✔      |    ✘    |         ✔          |
-| DSI Studio QA-Enhanced Tractography   |    ✔    |     ✘      |    ✘    |         ✘          |
-| Across-Software tractography          |    ✔    |     ✘      |    ✘    |         ✘          |
-| SIFT weighting                        |    ✔    |     ✘      |    ✘    |         ✔          |
-
 ## QC
 
 |                               | QSIPrep           | Tractoflow | PreQual | MRtrix3_connectome |
 | ----------------------------- | :---------------: | :--------: | :-----: | :----------------: |
 | Automated methods boilerplate |         ✔         |     ✘      |    ✘    |         ✘          |
 | HTML Preprocessing Report     | [NiWorkflows-based](preprocessing.html#visual-reports) |     ✘      | Custom  |      EddyQuad      |
-| HTML Reconstruction Report    | NiWorkflows-based |     ✘      | Custom  |         ✘          |
diff --git a/docs/conf.py b/docs/conf.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# qsiprep  documentation build configuration file, created by
+# QSIPrep documentation build configuration file, created by
 # sphinx-quickstart on Mon May  9 09:04:25 2016.
 #
 # This file is execfile()d with the current directory set to its
@@ -344,7 +344,7 @@
     (
         master_doc,
         "qsiprep",
-        "qsiprep Documentation",
+        "QSIPrep Documentation",
         author,
         "qsiprep",
         "One line description of project.",

diff --git a/docs/contributors.rst b/docs/contributors.rst
@@ -1,8 +1,8 @@
 .. include:: links.rst
 
-------------------------
-Contributing to qsiprep
-------------------------
+#########################
+Contributing to *QSIPrep*
+#########################
 
 This document explains how to prepare a new development environment and
 update an existing environment, as necessary.
@@ -14,8 +14,10 @@ By default, work should be built off of `pennlinc/qsiprep:latest
 installation_ guide for the basic procedure for running).
 
 
+*****************************
 Patching working repositories
-=============================
+*****************************
+
 In order to test new code without rebuilding the Docker image, it is
 possible to mount working repositories as source directories within the
 container.
@@ -54,8 +56,39 @@ Or you can patch Apptainer containers using the PYTHONPATH variable: ::
         /scratch/dataset /scratch/out participant -w /out/work/
 
 
+Running tests locally
+=====================
+
+To run the tests locally, *QSIRecon* includes a Python script to automatically mount the
+local clone into ``pennlinc/qsiprep:unstable`` and run tests with ``pytest``.
+The script will also download any required test data from Box.
+
+To run the tests, navigate to the tests folder and run ``run_local_tests.py``::
+
+    $ cd /path/to/qsiprep/qsiprep/tests
+    $ python run_local_tests.py
+
+You can select individual tests to run by using the ``-m`` (to select markers) or ``-k`` (the select tests by name) flags::
+
+    $ python run_local_tests.py -m "dsdti_fmap"
+    $ python run_local_tests.py -k "test_some_name"
+
+.. warning::
+
+    Please note that the integration tests in *QSIPrep* are computationally intensive
+    and may take a long time to run, so be prepared for that before running them on a laptop.
+
+
+If the tests pass, that's a good sign that your changes are solid.
+We also recommend opening the HTML reports produced by integration tests to check the results.
+Evaluating whether the HTML reports look "good" requires some domain knowledge and
+familiarity with *QSIPrep* outputs.
+
+
+*******************
 Adding dependencies
-===================
+*******************
+
 New dependencies to be inserted into the Docker image will either be
 Python or non-Python dependencies.
 Python dependencies may be added in three places, depending on whether
@@ -81,8 +114,11 @@ For example, installing an ``apt`` package may be done as follows: ::
     RUN apt-get update && \
         apt-get install -y <PACKAGE>
 
+
+***********************
 Rebuilding Docker image
-=======================
+***********************
+
 If it is necessary to rebuild the Docker image, a local image named
 ``qsiprep`` may be built from within the working qsiprep
 repository, located in ``~/projects/qsiprep``: ::
@@ -93,12 +129,13 @@ To work in this image, replace ``pennlinc/qsiprep:latest`` with
 ``qsiprep`` in any of the above commands.
 
 
+***********************************************
 Adding new features to the citation boilerplate
-===============================================
+***********************************************
 
 The citation boilerplate is built by adding two dunder attributes
 of workflow objects: ``__desc__`` and ``__postdesc__``.
-Once the full *qsiprep* workflow is built, starting from the
+Once the full *QSIPrep* workflow is built, starting from the
 outer workflow and visiting all sub-workflows in topological
 order, all defined ``__desc__`` are appended to the citation
 boilerplate before descending into sub-workflows.

diff --git a/docs/help.rst b/docs/help.rst
@@ -1,5 +1,6 @@
+############
 Getting help
-============
+############
 
 If you have a problem, would like to ask a question about how to use QSIprep,
 or have a question about the features and behavior of QSIprep, please submit

diff --git a/docs/index.rst b/docs/index.rst
@@ -1,13 +1,14 @@
-.. qsiprep documentation master file, created by
+.. QSIPrep documentation master file, created by
    sphinx-quickstart on Mon May  9 09:04:25 2016.
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
 .. include:: links.rst
 .. include:: ../README.rst
 
+########
 Contents
---------
+########
 
 .. toctree::
    :maxdepth: 3
@@ -18,7 +19,6 @@ Contents
    preprocessing
    reconstruction
    contributors
-   citing
    changes
    api
    comparisons