From 7ca422f9afea1171ac9ea863b02db2333c289eeb Mon Sep 17 00:00:00 2001 From: Cyril Pernet Date: Fri, 30 Jun 2023 14:43:44 +0200 Subject: [PATCH 01/13] define raw and derivatives.md follow the BIDS derivatives meeting --- src/derivatives/introduction.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index c18e8d5190..7c98492fd9 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -1,12 +1,19 @@ # BIDS Derivatives -Derivatives are outputs of common processing pipelines, capturing data and +Derivatives are outputs of common processing pipelines, capturing data and meta-data sufficient for a researcher to understand and (critically) reuse those -outputs in subsequent processing. +outputs in subsequent processing. Standardizing derivatives is motivated by use cases where formalized -machine-readable access to processed data enables higher level processing. +machine-readable access to processed data enables higher-level processing. + +The following sections cover additions to and divergences from "raw" BIDS. Raw data are +data that have been curated to BIDS, i.e. a BIDS “raw” dataset does not have another BIDS +dataset as a source. If you are in doubt if your dataset is "raw" or +is "derivatives" ask yourself the following question: "Are my data derived from at least +one BIDS valid dataset?" if the answer is yes, then this is a BIDS derivative dataset. + +## Derivatives storage and folders structure -The following sections cover additions to and divergences from "raw" BIDS. Placement and naming conventions for derived datasets are addressed in [Storage of derived datasets][storage], and dataset-level metadata is included in [Derived dataset and pipeline description][derived-dataset-description]. From 38f40f390123a6635f111caf18a9506dd545ebd4 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 30 Jun 2023 12:47:18 +0000 Subject: [PATCH 02/13] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- src/derivatives/introduction.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index 7c98492fd9..7258dbbccb 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -1,16 +1,16 @@ # BIDS Derivatives -Derivatives are outputs of common processing pipelines, capturing data and +Derivatives are outputs of common processing pipelines, capturing data and meta-data sufficient for a researcher to understand and (critically) reuse those -outputs in subsequent processing. +outputs in subsequent processing. Standardizing derivatives is motivated by use cases where formalized machine-readable access to processed data enables higher-level processing. -The following sections cover additions to and divergences from "raw" BIDS. Raw data are +The following sections cover additions to and divergences from "raw" BIDS. Raw data are data that have been curated to BIDS, i.e. a BIDS “raw” dataset does not have another BIDS dataset as a source. If you are in doubt if your dataset is "raw" or is "derivatives" ask yourself the following question: "Are my data derived from at least -one BIDS valid dataset?" if the answer is yes, then this is a BIDS derivative dataset. +one BIDS valid dataset?" if the answer is yes, then this is a BIDS derivative dataset. ## Derivatives storage and folders structure From 7dd3d17b440c7210755e6ac109dd5c66dc9059e9 Mon Sep 17 00:00:00 2001 From: Cyril Pernet Date: Fri, 30 Jun 2023 15:57:20 +0200 Subject: [PATCH 03/13] Update src/derivatives/introduction.md Co-authored-by: Yaroslav Halchenko --- src/derivatives/introduction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index 7258dbbccb..c6657d21cd 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -10,7 +10,7 @@ The following sections cover additions to and divergences from "raw" BIDS. Raw d data that have been curated to BIDS, i.e. a BIDS “raw” dataset does not have another BIDS dataset as a source. If you are in doubt if your dataset is "raw" or is "derivatives" ask yourself the following question: "Are my data derived from at least -one BIDS valid dataset?" if the answer is yes, then this is a BIDS derivative dataset. +one other valid BIDS dataset?" if the answer is yes, then this is a BIDS derivative dataset. ## Derivatives storage and folders structure From 90b887480736b3e9d80ddf13d4428bac931fe10c Mon Sep 17 00:00:00 2001 From: Cyril Pernet Date: Wed, 5 Jul 2023 20:56:25 +0200 Subject: [PATCH 04/13] Update introduction.md add examples --- src/derivatives/introduction.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index c6657d21cd..a1d0895f44 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -12,6 +12,18 @@ dataset as a source. If you are in doubt if your dataset is "raw" or is "derivatives" ask yourself the following question: "Are my data derived from at least one other valid BIDS dataset?" if the answer is yes, then this is a BIDS derivative dataset. +Examples: + +A defaced T1w image would typically be made during the curation process and is thus under raw + +`sourcedata/private/sub-01/anat/sub-01_T1w.nii.gz` +`sub-01/anat/sub-01_desc-defaced_T1w.nii.gz` + +A defaced T1w image could also, in theory, be derived from a BIDS dataset and would thus be under derivatives + +`sub-01/anat/sub-01_T1w.nii.gz` +`derivatives/sub-01/anat/sub-01_desc-defaced_T1w.nii.gz` + ## Derivatives storage and folders structure Placement and naming conventions for derived datasets are addressed in From 32b6cd4b0b2441bd2c087fa6a1ba50e6cf3ea787 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 5 Jul 2023 18:56:45 +0000 Subject: [PATCH 05/13] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- src/derivatives/introduction.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index a1d0895f44..bcfab302c5 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -16,12 +16,12 @@ Examples: A defaced T1w image would typically be made during the curation process and is thus under raw -`sourcedata/private/sub-01/anat/sub-01_T1w.nii.gz` +`sourcedata/private/sub-01/anat/sub-01_T1w.nii.gz` `sub-01/anat/sub-01_desc-defaced_T1w.nii.gz` A defaced T1w image could also, in theory, be derived from a BIDS dataset and would thus be under derivatives -`sub-01/anat/sub-01_T1w.nii.gz` +`sub-01/anat/sub-01_T1w.nii.gz` `derivatives/sub-01/anat/sub-01_desc-defaced_T1w.nii.gz` ## Derivatives storage and folders structure From 64bd1c811e42524146d48b33c7e6a4b788b955f3 Mon Sep 17 00:00:00 2001 From: Cyril Pernet Date: Mon, 10 Jul 2023 14:39:38 +0200 Subject: [PATCH 06/13] Update src/derivatives/introduction.md Co-authored-by: Chris Markiewicz --- src/derivatives/introduction.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index bcfab302c5..1a2bd1189a 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -21,8 +21,10 @@ A defaced T1w image would typically be made during the curation process and is t A defaced T1w image could also, in theory, be derived from a BIDS dataset and would thus be under derivatives -`sub-01/anat/sub-01_T1w.nii.gz` -`derivatives/sub-01/anat/sub-01_desc-defaced_T1w.nii.gz` +``` +sub-01/anat/sub-01_T1w.nii.gz +derivatives/sub-01/anat/sub-01_desc-defaced_T1w.nii.gz +``` ## Derivatives storage and folders structure From 583590a83863e704fcb6877d8017871cef2be578 Mon Sep 17 00:00:00 2001 From: Cyril Pernet Date: Mon, 10 Jul 2023 14:39:57 +0200 Subject: [PATCH 07/13] Update src/derivatives/introduction.md Co-authored-by: Chris Markiewicz --- src/derivatives/introduction.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index 1a2bd1189a..9b3b61dd49 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -6,11 +6,9 @@ outputs in subsequent processing. Standardizing derivatives is motivated by use cases where formalized machine-readable access to processed data enables higher-level processing. -The following sections cover additions to and divergences from "raw" BIDS. Raw data are -data that have been curated to BIDS, i.e. a BIDS “raw” dataset does not have another BIDS -dataset as a source. If you are in doubt if your dataset is "raw" or -is "derivatives" ask yourself the following question: "Are my data derived from at least -one other valid BIDS dataset?" if the answer is yes, then this is a BIDS derivative dataset. +The following sections cover additions to and divergences from "raw" BIDS. +Raw data are data that have been curated into BIDS from a non-BIDS source. +If a dataset is derived from at least one other valid BIDS dataset, then it is a derivative dataset. Examples: From 882a9d6b7767163ba3475580a02d0c8eff7f313b Mon Sep 17 00:00:00 2001 From: Chris Markiewicz Date: Tue, 11 Jul 2023 10:30:40 -0400 Subject: [PATCH 08/13] FIX: Update code fences --- src/derivatives/introduction.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index 9b3b61dd49..a8662d67bd 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -14,12 +14,14 @@ Examples: A defaced T1w image would typically be made during the curation process and is thus under raw -`sourcedata/private/sub-01/anat/sub-01_T1w.nii.gz` -`sub-01/anat/sub-01_desc-defaced_T1w.nii.gz` +```Text +sourcedata/private/sub-01/anat/sub-01_T1w.nii.gz +sub-01/anat/sub-01_T1w.nii.gz +``` A defaced T1w image could also, in theory, be derived from a BIDS dataset and would thus be under derivatives -``` +```Text sub-01/anat/sub-01_T1w.nii.gz derivatives/sub-01/anat/sub-01_desc-defaced_T1w.nii.gz ``` From 915aa3eb46c79f916fe992606dc6d7e8156eb334 Mon Sep 17 00:00:00 2001 From: Chris Markiewicz Date: Tue, 11 Jul 2023 10:32:42 -0400 Subject: [PATCH 09/13] FIX: folders -> directory For consistency --- src/derivatives/introduction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/derivatives/introduction.md b/src/derivatives/introduction.md index a8662d67bd..9cad29a14a 100644 --- a/src/derivatives/introduction.md +++ b/src/derivatives/introduction.md @@ -26,7 +26,7 @@ sub-01/anat/sub-01_T1w.nii.gz derivatives/sub-01/anat/sub-01_desc-defaced_T1w.nii.gz ``` -## Derivatives storage and folders structure +## Derivatives storage and directory structure Placement and naming conventions for derived datasets are addressed in [Storage of derived datasets][storage], and dataset-level metadata is included From 01be6848e7eeecba175f9aa3e69889c32bb02445 Mon Sep 17 00:00:00 2001 From: bids-maintenance Date: Mon, 24 Jul 2023 15:52:32 +0000 Subject: [PATCH 10/13] [DOC] Auto-generate changelog entry for PR #1537 --- src/CHANGES.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/CHANGES.md b/src/CHANGES.md index 261bb25a44..f6b0a97cd4 100644 --- a/src/CHANGES.md +++ b/src/CHANGES.md @@ -3,6 +3,7 @@ ## [Unreleased](https://github.com/bids-standard/bids-specification/tree/HEAD) - \[FIX] Update links to BIDS examples [#1545](https://github.com/bids-standard/bids-specification/pull/1545) ([Remi-Gau](https://github.com/Remi-Gau)) +- \[ENH] Clarify meaning of raw vs derivative datasets [#1537](https://github.com/bids-standard/bids-specification/pull/1537) ([CPernet](https://github.com/CPernet)) - \[FIX] match subject label in folder and filename in func example [#1536](https://github.com/bids-standard/bids-specification/pull/1536) ([Remi-Gau](https://github.com/Remi-Gau)) - FIX: Trail MEG directory formats with `/` [#1534](https://github.com/bids-standard/bids-specification/pull/1534) ([effigies](https://github.com/effigies)) - \[ENH] Permit CITATION.cff as structured alternative to some dataset_description fields [#1525](https://github.com/bids-standard/bids-specification/pull/1525) ([effigies](https://github.com/effigies)) From 92837cdebd783f3997cfa0d37b223f404ddd6d19 Mon Sep 17 00:00:00 2001 From: Stefan Appelhoff Date: Mon, 31 Jul 2023 16:27:38 +0200 Subject: [PATCH 11/13] Update `templates` in modality-agnostic-files (#1564) --- src/modality-agnostic-files.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/modality-agnostic-files.md b/src/modality-agnostic-files.md index fda6643276..7bc09af991 100644 --- a/src/modality-agnostic-files.md +++ b/src/modality-agnostic-files.md @@ -5,7 +5,8 @@ Templates: - `dataset_description.json` -- `README` +- `README[.md|.rst|.txt]` +- `CITATION.cff` - `CHANGES` - `LICENSE` From d6e77f18fcda62ecf55265e7bf494a037ea494c4 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 1 Aug 2023 10:47:16 +0200 Subject: [PATCH 12/13] [pre-commit.ci] pre-commit autoupdate (#1568) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/pyCQA/flake8: 6.0.0 → 6.1.0](https://github.com/pyCQA/flake8/compare/6.0.0...6.1.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 53baab42b6..a2ef4cf862 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -33,7 +33,7 @@ repos: files: tools/schemacode args: ["--settings-file", "tools/schemacode/pyproject.toml"] - repo: https://github.com/pyCQA/flake8 - rev: 6.0.0 + rev: 6.1.0 hooks: - id: flake8 args: [--config=tools/schemacode/setup.cfg] From d3906e2863136fc0ff4e6ed7952ff2a7c60c7c8d Mon Sep 17 00:00:00 2001 From: Ariel Rokem Date: Thu, 3 Aug 2023 11:03:27 -0700 Subject: [PATCH 13/13] Adds Kim Ray as maintainer. (#1571) --- DECISION-MAKING.md | 1 + 1 file changed, 1 insertion(+) diff --git a/DECISION-MAKING.md b/DECISION-MAKING.md index 05e4c91887..88474ac82f 100644 --- a/DECISION-MAKING.md +++ b/DECISION-MAKING.md @@ -30,6 +30,7 @@ Current and past members of the steering group can be found | Eric Earl ([@ericearl](https://github.com/ericearl)) | 2h/week | | Dec 2021 | | Christine Rogers ([@christinerogers](https://github.com/christinerogers)) | 2h/mo | Interoperability, EEG and multi-modal | Apr 2023 | | Nell Hardcastle ([@nellh](https://github.com/nellh)) | 2h/week | | Jul 2023 | +| Kimberly Ray ([@KimberlyLRay](https://github.com/KimberlyLRay)) | 1h/week | | Nov 2022 | In addition to the [BIDS Governance](https://bids.neuroimaging.io/governance.html#bids-maintainers-group) classification of a maintainer, maintainers may declare a limited scope of responsibility.