Merge pull request #340 from VisLab/develop

First pass at a demo dataset

datasets/eeg_ds003645s_hed_column/task-FacePerception_events.json

@@ -17,8 +17,7 @@
       "show_cross": "Display only a white cross to mark start of trial and fixation.",
       "left_press": "Experiment participant presses a key with left index finger.",
       "right_press": "Experiment participant presses a key with right index finger.",
-      "setup_left_sym": "Setup for experiment with pressing key with left index finger means a face with above average symmetry.",
-      "setup_right_sym": "Setup for experiment with pressing key with right index finger means a face with above average symmetry.",
+
       "double_press": "Experiment participant presses both keys ."
     }
   },

datasets/eeg_ds003645s_hed_demo/README.md

@@ -0,0 +1,135 @@
+## Introduction
+
+Hierarchical Event Descriptors (HED) is BIDS' mechanism for providing 
+dataset annotations and metadata in a machine-actionable way.
+This means that tools can extract annotations automatically for analysis.
+The purpose of the dataset is to demonstrate how to use HED in various ways.
+BIDS allows HED to be used in any
+[BIDS tabular file](https://bids-specification.readthedocs.io/en/stable/common-principles.html#tabular-files) (`.tsv`).
+
+### What's in this dataset
+
+This demo dataset is derived from a number of other datasets,
+but particularly `ds003645` from openNeuro as well as the
+`fnirs_automaticity`, `micro_SPIM`, and `motion_dualtask` datasets
+from the [bids-examples](https://github.com/bids-standard/bids-examples)
+GitHub repository.
+
+| Subject | Session | Modalities illustrated     |
+| ------- |---------|----------------------------|  
+| sub-002 |         | eeg, beh, phenotype, scans | 
+| sub-003 |         | eeg, phenotype, scans |
+| sub-004 | ses-1   | eeg, anat, beh, micr, phenotype, scans |
+| sub-004 | ses-2   | eeg, motion, phenotype, scans | 
+
+
+### HED-supported files
+
+HED supports these BIDS `.tsv` (and accompanying `.json`) files.
+In all cases, HED automatically combines annotations in a `HED`
+column of the `.tsv` file with annotations of column values
+in the corresponding `.json` file.
+
+
+| File                                | Use | 
+|-------------------------------------|-----| 
+| [`participants.tsv`](#participants) | Characteristics of participants. |
+| `_events.tsv`                       | Descriptions of events in data. |
+| `_scans.tsv`                        | Recording-wide experimental conditions, <br/>Experiment setup, start time, notes on a recording. |
+| `_beh.tsv`                          | Responses to behavioral tasks. |
+| `_channels.tsv`                     | Meaning of user-defined fields and notes about channels. |
+| `samples.tsv`                       | Characteristics of samples associated with the dataset. |
+| `phenotypes/`                       | Responses to questionnaires, medical and other information. |
+
+This demo dataset illustrates HED's use in the above types of files.
+The `.json` file contains two types of HED annotations.
+**Categorical** annotations provide individual HED strings for each
+unique column value. **Value** annotations provide a single
+HED string with a `#` placeholder for the entire column.
+When the annotations for a row of the `.tsv` file are assembled,
+the column value is substituted for that placeholder.
+
+Currently, HED ignores tabular files that correspond to continuous time series
+and do not have column names
+(e.g., `_motion.tsv`, `_physio.tsv`, and `_stim.tsv`).
+
+### Participants
+HED-annotated subject information contained in the BIDS-required 
+[`participants.tsv`](https://bids-specification.readthedocs.io/en/stable/modality-agnostic-files.html#participants-file)
+and its accompanying `participants.json` file can be extracted
+as a HED string and then used in analysis for search or extracting design
+matrices and contrasts.
+
+The demo [`participants.json`](./participants.json) is:
+
+```json
+{
+  "participant_id": {
+    "LongName": "Participant identifier",
+    "Description": "Unique dataset subject identifier",
+    "HED": "(Experiment-participant, ID/#)"
+  },
+  "sex": {
+    "Description": "Sex of the subject",
+    "Levels": {
+      "M": "male",
+      "F": "female"
+    },
+    "HED": {
+      "M": "Male",
+      "F": "Female"
+    }
+  },
+  "age": {
+    "Description": "Age of the subject",
+    "Units": "years",
+    "HED": "Age/#"
+  }
+}
+```
+
+The `participant_id` and `age` column are annotated as value columns,
+while `sex` is annotated as a value column.
+
+In the demo, the first row of the [`partipants.tsv`](./participants.tsv) file is:
+
+| participant_id | age | sex | HED |
+| -------------- | --- | --- | --- |
+| sub-002 | 31 | M | Healthy,Rested,Novice-level |
+
+At validator or analysis time, the annotations for the columns of a `.tsv`
+file are concatenated for a row in a comma-separated string. 
+The HED annotation for the first row of the demo `participants.tsv` file is:
+
+```code 
+"(Experiment-participant, ID/sub-001),Age/3,Male,Healthy,Rested,Novide-level"
+```
+
+This annotation can then be used in downstream tools during analysis.
+
+The parentheses in HED strings are meant to applicable modifiers with
+the item being modified, in this case `ID/sub-001`.
+
+We can use the HED curly brace notation to get a more desirable grouping.
+If the HED annotation for the `participant_id` given in `participants.json` were:
+
+```json
+{
+    "participant_id": {
+        "LongName": "Participant identifier",
+        "Description": "Unique dataset subject identifier",
+        "HED": "(Experiment-participant, ID/#, {sex}, {age}, {HED})"
+    }
+}
+```
+
+then the `participant_id` annotation is treated like a template.
+On assembly, the annotations for the `sex`, `age`, and `HED` columns
+are inserted into the template, rather than concatenated:
+
+```code
+"(Experiment-participant, ID/sub-001),Age/3,Male,Healthy,Rested,Novide-level"
+```
+
+See [Assembly and curly braces](https://www.hed-resources.org/en/latest/HedAnnotationQuickstart.html#assembly-and-curly-braces)
+in the [HED annotation quickstart](https://www.hed-resources.org/en/latest/HedAnnotationQuickstart.html#) for more information.

datasets/eeg_ds003645s_hed_demo/participants.json

@@ -1,7 +1,8 @@
 {
   "participant_id": {
     "LongName": "Participant identifier",
-    "Description": "Unique subject identifier"
+    "Description": "Unique dataset subject identifier",
+    "HED": "(Experiment-participant, ID/#)"
   },
   "sex": {
     "Description": "Sex of the subject",
@@ -16,6 +17,7 @@
   },
   "age": {
     "Description": "Age of the subject",
-    "Units": "years"
+    "Units": "years",
+    "HED": "Age/#"
   }
 }

datasets/eeg_ds003645s_hed_demo/participants.tsv

@@ -1,3 +1,4 @@
 participant_id	age	sex	HED
 sub-002	31	M	Healthy,Rested,Novice-level
 sub-003	25	M	Healthy,Drowsy,Expert-level
+sub-004	30	M	Expert-level

datasets/eeg_ds003645s_hed_demo/phenotype/KSSSleep.json

@@ -0,0 +1,41 @@
+{
+  "participant_id": {
+    "Description": "Dataset-wide identifier of participant",
+    "HED": "((Experiment-participant, ID/#), {session_id}, {kss_scale}, {notes})"
+  },
+  "session_id": {
+    "Description": "The number of the session in this dataset for this participant.",
+    "HED": "(Label/Session, ID/#)"
+  },
+  "kss_scale": {
+    "Description": "Karolinska sleepiness",
+    "Levels": {
+      "1": "Extremely alert",
+      "2": "Very alert",
+      "3": "Alert",
+      "4": "Rather alert",
+      "5": "Neither alert nor sleepy",
+      "6": "Some signs of sleepiness",
+      "7": "Sleepy but no effort to keep awake",
+      "8": "Sleepy, but some effort to keep awake",
+      "9": "Very sleepy, great effort to keep awake, fighting sleep",
+      "10": "Extremely sleepy, can't keep awake"
+    },
+    "HED": {
+      "1": "(Alert, High)",
+      "2": "(Alert, Medium)",
+      "3": "Alert",
+      "4": "(Alert, Low)",
+      "5": "Awake",
+      "6": "(Drowsy, Low)",
+      "7": "Drowsy",
+      "8": "(Drowsy, Medium)",
+      "9": "(Drowsy, High)",
+      "10": "Asleep"
+    }
+  },
+  "notes": {
+    "Description": "additional comments/observations/notes, reported by the participant",
+    "HED": "Experimental-note/#"
+  }
+}

datasets/eeg_ds003645s_hed_demo/phenotype/KSSSleep.tsv

@@ -0,0 +1,5 @@
+participant_id	session_id	kss_scale	notes
+sub-002	n/a	1	Highly caffeinated.
+sub-003	n/a	4	Normal sleep.
+sub-004	ses-1	7	Bad dreams.
+sub-004	ses-2	9	Subject stayed up all night.

datasets/eeg_ds003645s_hed_demo/phenotype/trainLog.json

@@ -0,0 +1,33 @@
+{
+  "trainLog": {
+    "Description": "logbook filled in by the participants after each training session"
+  },
+  "train_num": {
+    "Description": "The number of this practice session",
+    "HED": "(Item-count/#, Label/Train-session)"
+  },
+  "duration_finger": {
+    "Description": "reported practice time for the finger tapping task",
+    "Units": "minutes",
+    "HED": "(Duration/# minute, (Move, (Finger)))"
+  },
+  "duration_foot": {
+    "Description": "reported practice time for the foot stepping task",
+    "Units": "minutes",
+    "HED": "(Duration/# minute, (Move, (Foot)))"
+  },
+  "assess_finger": {
+    "Description": "'On a scale from 1 to 10, how would rate your performance today?' for the finger tapping task",
+    "Range": "1 (can't perform the sequence at all) to 10 (automaticity is reached)",
+    "HED": "(Finger, Categorical-level-value, Label/#)"
+  },
+  "assess_foot": {
+    "Description": "'On a scale from 1 to 10, how would rate your performance today?' for the foot stepping task",
+    "Range": "1 (can't perform the sequence at all) to 10 (automaticity is reached)",
+    "HED": "(Foot, Categorical-level-value, Label/#)"
+  },
+  "notes": {
+    "Description": "additional comments/observations/notes, reported by the participant",
+    "HED": "Experimental-note/#"
+  }
+}

datasets/eeg_ds003645s_hed_demo/phenotype/trainLog.tsv

@@ -0,0 +1,10 @@
+participant_id	train_num	duration_finger	duration_foot	assess_finger	assess_foot	notes
+sub-002	1	5	4	7	5	i hate the foot stomping so much.
+sub-002	2	5	5	8	6	able to watch tv in the meantime.
+sub-003	1	3	5	9	6	Missed one day but still going strong.
+sub-003	2	10	10	7	n/a	n/a
+sub-003	3	5	5	3	4	n/a
+sub-004	1	5	5	5	4	I did practice but forgot to fill in the form.
+sub-004	2	5	5	5	5	The sequences now go almost automatic.
+sub-004	3	5	5	6	6	n/a
+sub-004	4	n/a	n/a	1	1	n/a

datasets/eeg_ds003645s_hed_demo/samples.json

@@ -0,0 +1,14 @@
+{
+    "sample_id": {
+        "Description": "Sample ID",
+        "HED": "((ID/#, Label/Sample-id), {participant_id}, {sample_type})"
+    },
+    "participant_id": {
+        "Description": "Participant ID from whom tissue samples have been acquired",
+        "HED": "(ID/#, Experiment-participant)"
+    },
+    "sample_type": {
+        "Description": "Type of sample from ENCODE Biosample Type (https://www.encodeproject.org/profiles/biosample_type)",
+        "HED": "Label/#"
+    }
+}

datasets/eeg_ds003645s_hed_demo/samples.tsv

@@ -0,0 +1,3 @@
+sample_id	participant_id	sample_type
+sample-A	sub-004	tissue
+sample-B	sub-004	tissue