Bring outputs up to date with BEPs 11, 12, 17, and 38

docs/outputs.rst

@@ -9,23 +9,25 @@ The  *XCP-D* outputs are written out in BIDS format and consist of three main pa
 
 .. admonition:: A note on BIDS compliance
 
-   *XCP-D* attempts to follow the BIDS specification as best as possible.
+   *XCP-D* attempts to follow the BIDS specification as closely as possible.
    However, many *XCP-D* derivatives are not currently covered by the specification.
    In those instances, we attempt to follow recommendations from existing BIDS Extension Proposals
    (BEPs), which are in-progress proposals to add new features to BIDS.
+   However, we do not guarantee compliance with any BEP,
+   as they are not yet part of the official BIDS specification.
 
-   Three BEPs that are of particular use in *XCP-D* are
+   Four BEPs that are of particular use in *XCP-D* are
+   `BEP011: Structural preprocessing derivatives <https://github.com/bids-standard/bids-specification/pull/518>`_,
    `BEP012: Functional preprocessing derivatives <https://github.com/bids-standard/bids-specification/pull/519>`_,
-   `BEP017: BIDS connectivity matrix data schema <https://docs.google.com/document/d/1ugBdUF6dhElXdj3u9vw0iWjE6f_Bibsro3ah7sRV0GA/edit?usp=sharing>`_,
+   `BEP017: Relationship & connectivity matrix data schema <https://docs.google.com/document/d/1ugBdUF6dhElXdj3u9vw0iWjE6f_Bibsro3ah7sRV0GA/edit?usp=sharing>`_,
    and
-   `BEP038: Atlas Specification <https://docs.google.com/document/d/1RxW4cARr3-EiBEcXjLpSIVidvnUSHE7yJCUY91i5TfM/edit?usp=sharing>`_
-   (currently unnumbered).
+   `BEP038: Atlas Specification <https://docs.google.com/document/d/1RxW4cARr3-EiBEcXjLpSIVidvnUSHE7yJCUY91i5TfM/edit?usp=sharing>`_.
 
    In cases where a derivative type is not covered by an existing BEP,
    we have simply attempted to follow the general principles of BIDS.
 
-   If you discover a problem with the BIDS compliance of *XCP-D*'s derivatives, please open an
-   issue in the *XCP-D* repository.
+   If you discover a problem with the BIDS compliance of *XCP-D*'s derivatives,
+   please open an issue in the *XCP-D* repository.
 
 
 ***************
@@ -86,15 +88,16 @@ Atlases are written out to the ``atlases`` subfolder, following BEP038.
 
    xcp_d/
       atlases/
+         dataset_description.json
          atlas-<label>/
             atlas-<label>_dseg.json
             atlas-<label>_dseg.tsv
 
             # NIfTI
-            space-<label>_atlas-<label>_dseg.nii.gz
+            atlas-<label>_space-<label>_dseg.nii.gz
 
             # CIFTI
-            space-<label>_atlas-<label>_dseg.dlabel.nii
+            atlas-<label>_space-<label>_dseg.dlabel.nii
 
 
 ******************
@@ -155,9 +158,9 @@ atlases it uses to parcellate the functional outputs.
    xcp_d/
       sub-<label>/[ses-<label>/]
          anat/
-            <source_entities>_space-fsLR_atlas-<atlas>_den-32k_desc-curv_morph.tsv
-            <source_entities>_space-fsLR_atlas-<atlas>_den-32k_desc-sulc_morph.tsv
-            <source_entities>_space-fsLR_atlas-<atlas>_den-32k_desc-thickness_morph.tsv
+            <source_entities>_space-fsLR_seg-<label>_den-32k_stat-mean_desc-curv_morph.tsv
+            <source_entities>_space-fsLR_seg-<label>_den-32k_stat-mean_desc-sulc_morph.tsv
+            <source_entities>_space-fsLR_seg-<label>_den-32k_stat-mean_desc-thickness_morph.tsv
 
 
 ******************
@@ -208,15 +211,33 @@ Denoised or residual BOLD data
    and primarily exist for compatibility with DCAN-specific analysis tools.
 
 The sidecar json files contains parameters of the data and processing steps.
+The Sources field contains BIDS URIs pointing to the files used to create the derivative.
+The associated DatasetLinks are defined in the dataset_description.json.
 
    .. code-block:: json-object
 
       {
-         "Freq Band": [0.01, 0.08],
-         "RepetitionTime": 2.0,
-         "compression": true,
-         "dummy vols": 0,
-         "nuisance parameters": "27P",
+         "EchoTime": 0.0424,
+         "EffectiveEchoSpacing": 0.000639989,
+         "FlipAngle": 51,
+         "Manufacturer": "Siemens",
+         "ManufacturersModelName": "Skyra",
+         "NuisanceParameters": "gsr_only",
+         "PhaseEncodingDirection": "j-",
+         "RepetitionTime": 3,
+         "SoftwareFilters": {
+            "Bandpass filter": {
+               "Filter order": 2,
+               "High-pass cutoff (Hz)": 0.01,
+               "Low-pass cutoff (Hz)": 0.08
+            }
+         },
+         "Sources": [
+            "bids:preprocessed:sub-0000001/ses-01/func/sub-0000001_ses-01_task-rest_space-MNI152NLin6Asym_desc-preproc_bold.nii.gz",
+            "bids:xcp_d:sub-0000001/ses-01/func/sub-0000001_ses-01_task-rest_outliers.tsv",
+            "bids:xcp_d:sub-0000001/ses-01/func/sub-0000001_ses-01_task-rest_desc-preproc_design.tsv"
+         ],
+         "TaskName": "resting state"
       }
 
 
@@ -235,19 +256,19 @@ This includes the atlases used to extract the timeseries.
       sub-<label>/[ses-<label>/]
          func/
             # NIfTI
-            <source_entities>_space-<label>_atlas-<label>_coverage.tsv
-            <source_entities>_space-<label>_atlas-<label>_timeseries.tsv
-            <source_entities>_space-<label>_atlas-<label>_measure-pearsoncorrelation_conmat.tsv
-            <source_entities>_space-<label>_atlas-<label>_measure-pearsoncorrelation_desc-<INT>volumes_conmat.tsv
+            <source_entities>_space-<label>_seg-<label>_stat-coverage_bold.tsv
+            <source_entities>_space-<label>_seg-<label>_stat-mean_timeseries.tsv
+            <source_entities>_space-<label>_seg-<label>_stat-pearsoncorrelation_relmat.tsv
+            <source_entities>_space-<label>_seg-<label>_stat-pearsoncorrelation_desc-<INT>volumes_relmat.tsv
 
             # CIFTI
-            <source_entities>_space-fsLR_atlas-<label>_den-91k_coverage.tsv
-            <source_entities>_space-fsLR_atlas-<label>_den-91k_coverage.pscalar.nii
-            <source_entities>_space-fsLR_atlas-<label>_den-91k_timeseries.tsv
-            <source_entities>_space-fsLR_atlas-<label>_den-91k_timeseries.ptseries.nii
-            <source_entities>_space-fsLR_atlas-<label>_den-91k_measure-pearsoncorrelation_conmat.tsv
-            <source_entities>_space-fsLR_atlas-<label>_den-91k_measure-pearsoncorrelation_conmat.pconn.nii
-            <source_entities>_space-fsLR_atlas-<label>_den-91k_measure-pearsoncorrelation_desc-<INT>volumes_conmat.tsv
+            <source_entities>_space-fsLR_seg-<label>_den-91k_stat-coverage_bold.tsv
+            <source_entities>_space-fsLR_seg-<label>_den-91k_stat-coverage_boldmap.pscalar.nii
+            <source_entities>_space-fsLR_seg-<label>_den-91k_stat-mean_timeseries.tsv
+            <source_entities>_space-fsLR_seg-<label>_den-91k_stat-mean_timeseries.ptseries.nii
+            <source_entities>_space-fsLR_seg-<label>_den-91k_stat-pearsoncorrelation_relmat.tsv
+            <source_entities>_space-fsLR_seg-<label>_den-91k_stat-pearsoncorrelation_boldmap.pconn.nii
+            <source_entities>_space-fsLR_seg-<label>_den-91k_stat-pearsoncorrelation_desc-<INT>volumes_relmat.tsv
 
 
 Resting-state metric derivatives (ReHo and ALFF)
@@ -273,18 +294,18 @@ data.
       sub-<label>/[ses-<label>/]
          func/
             # NIfTI
-            <source_entities>_space-<label>_reho.nii.gz
-            <source_entities>_space-<label>_alff.nii.gz
-            <source_entities>_space-<label>_desc-smooth_alff.nii.gz
-            <source_entities>_space-<label>_atlas-<atlas>_alff.tsv
-            <source_entities>_space-<label>_atlas-<atlas>_reho.tsv
+            <source_entities>_space-<label>_stat-reho_boldmap.nii.gz
+            <source_entities>_space-<label>_stat-alff_boldmap.nii.gz
+            <source_entities>_space-<label>_stat-alff_desc-smooth_boldmap.nii.gz
+            <source_entities>_space-<label>_seg-<label>_stat-alff_bold.tsv
+            <source_entities>_space-<label>_seg-<label>_stat-reho_bold.tsv
 
             # CIFTI
-            <source_entities>_space-fsLR_den-91k_reho.dscalar.nii
-            <source_entities>_space-fsLR_den-91k_alff.dscalar.nii
-            <source_entities>_space-fsLR_den-91k_desc-smooth_alff.dscalar.nii
-            <source_entities>_space-fsLR_atlas-<atlas>_alff.tsv
-            <source_entities>_space-fsLR_atlas-<atlas>_reho.tsv
+            <source_entities>_space-fsLR_den-91k_stat-reho_boldmap.dscalar.nii
+            <source_entities>_space-fsLR_den-91k_stat-alff_boldmap.dscalar.nii
+            <source_entities>_space-fsLR_den-91k_stat-alff_desc-smooth_boldmap.dscalar.nii
+            <source_entities>_space-fsLR_seg-<label>_stat-alff_bold.tsv
+            <source_entities>_space-fsLR_seg-<label>_stat-reho_bold.tsv
 
 
 Other outputs include quality control, framewise displacement, and confounds files
@@ -300,12 +321,7 @@ Other outputs include quality control, framewise displacement, and confounds fil
             <source_entities>[_desc-filtered]_motion.tsv
             <source_entities>_outliers.tsv
             <source_entities>_design.tsv
-
-            # NIfTI
-            <source_entities>_space-<label>_desc-linc_qc.csv
-
-            # CIFTI
-            <source_entities>_space-fsLR_desc-linc_qc.csv
+            <source_entities>_space-<label>_desc-linc_qc.tsv
 
 ``[desc-filtered]_motion.tsv`` is a tab-delimited file with seven columns:
 one for each of the six filtered motion parameters, as well as "framewise_displacement".

docs/usage.rst

@@ -419,16 +419,16 @@ generate the time series:
       -cifti-parcellate \
       {SUB}_ses-{SESSION}_task-{TASK}_run-{RUN}_space-fsLR_den-91k_desc-residual_bold.dtseries.nii \
       your_parcels.dlabel \
-      {SUB}_ses-{SESSION}_task-{TASK}_run-{RUN}_space-fsLR_den-91k_desc-residual_bold.ptseries.nii
+      {SUB}_ses-{SESSION}_task-{TASK}_run-{RUN}_space-fsLR_den-91k_desc-residual_timeseries.ptseries.nii
 
 After this, if one wishes to have a connectivity matrix:
 
 .. code-block:: bash
 
    wb_command \
       -cifti-correlation \
-      {SUB}_ses-{SESSION}_task-{TASK}_run-{RUN}_space-fsLR_den-91k_desc-residual_bold.ptseries.nii \
-      {SUB}_ses-{SESSION}_task-{TASK}_run-{RUN}_space-fsLR_den-91k_desc-residual_bold.pconn.nii
+      {SUB}_ses-{SESSION}_task-{TASK}_run-{RUN}_space-fsLR_den-91k_desc-residual_timeseries.ptseries.nii \
+      {SUB}_ses-{SESSION}_task-{TASK}_run-{RUN}_space-fsLR_den-91k_desc-residual_boldmap.pconn.nii
 
 More information can be found at the HCP
 `documentation <https://www.humanconnectome.org/software/workbench-command>`_.

xcp_d/cli/combineqc.py

@@ -33,17 +33,17 @@ def main(args=None):
     opts = get_parser().parse_args(args)
 
     xcpd_dir = os.path.abspath(opts.xcpd_dir)
-    outputfile = os.path.join(os.getcwd(), f"{opts.output_prefix}_allsubjects_qc.csv")
+    outputfile = os.path.join(os.getcwd(), f"{opts.output_prefix}_allsubjects_qc.tsv")
 
     qc_files = []
     for dirpath, _, filenames in os.walk(xcpd_dir):
         for filename in filenames:
-            if filename.endswith("_desc-linc_qc.csv"):
+            if filename.endswith("_desc-linc_qc.tsv"):
                 qc_files.append(os.path.join(dirpath, filename))
 
-    dfs = [pd.read_csv(qc_file) for qc_file in qc_files]
+    dfs = [pd.read_table(qc_file) for qc_file in qc_files]
     df = pd.concat(dfs, axis=0)
-    df.to_csv(outputfile, index=False)
+    df.to_csv(outputfile, index=False, sep="\t")
 
 
 if __name__ == "__main__":