[SCHEMA] Add full object definitions for valid values in schema (#919)

* Add some descriptions. Mostly just TODOs for now.

* Start rendering value restrictions in descriptions.

* Fix.

* Revert rendering. Will move to separate PR.

* Run black.

* Add more enum descriptions.

* Add nonstandard and non-template coordinate systems.

I'm not 100% sure about this one. I grabbed values from the coordinate systems appendix.

* Fix linting issues.

* Run prettier.

* Make the expanded enums into objects.

* Fix up rendering, hopefully.

* Update metadata.yaml

* Update render.py

* whoops.

* Update render.py

* Whoops!

* Finish resolving conflicts.

* Move enums to a separate file.

* Fix on and off.

* Remove duplicate keys.

* Start adding channel type values.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add EEG columns.

* Remove empty entries.

* Add MEG channels.

* Add NIRS channels.

* Try fixing the formatting.

* Fix.

* OTHER sex isn't the same as OTHER channel type.

* Consolidate channels enums across datatypes.

* Replace name with value.

* Replicate #1478.

* Add .value to value refs.

* Add tags to channel values.

* ENH: Allow for dereferencing individual values in schema

* Rename values to enum.

* Rename enum to enum_values.

* Fix test.

* Rename and move enums around.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update columns.yaml

* Document new sub-namespace.

* Fix table.

* Remove unused enum groups.

* Apply suggestions from code review

Co-authored-by: Chris Markiewicz <[email protected]>

* Update the other direction enum defs.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Stefan Appelhoff <[email protected]>

src/schema/README.md

@@ -336,7 +336,7 @@ or whether objects are required in a given dataset or file.
 
 ### Overview
 
-There are currently 11 sub-namespaces, which fall into five rough categories.
+There are currently 12 sub-namespaces, which fall into five rough categories.
 
 The namespaces are:
 
@@ -352,6 +352,7 @@ The namespaces are:
 | `objects.extensions`        | Filename component that describe the format of the file                             | Value terms      |
 | `objects.formats`           | Terms that define the forms values (for example, in metadata) might take            | Formats          |
 | `objects.files`             | Files and directories that may appear at the root of a dataset                      | Files            |
+| `objects.enums`             | Full descriptions of enumerated values used in other sub-namespaces                 | Value terms      |
 
 Because these objects vary, the contents of each namespace can vary.
 
@@ -554,6 +555,13 @@ The convention can be summed up in the following rules:
     | `description`  | Term definition                                                                      |
     | `file_type`    | Indicator that the file is a regular file (`"regular"`) or directory (`"directory"`) |
 
+-   `objects.enums`
+    | Field          | Description            |
+    | -------------- | ---------------------- |
+    | `display_name` | Human-friendly name    |
+    | `description`  | Term definition        |
+    | `value`        | String value of `enum` |
+
 ## Rule files
 
 The `rules.*` namespace contains most of the validatable content of the schema,
@@ -910,11 +918,11 @@ EEGChannels:
   - extension == ".tsv"
   initial_columns:
   - name__channels
-  - type__eeg_channels
+  - type__channels
   - units
   columns:
     name__channels: required
-    type__eeg_channels: required
+    type__channels: required
     units: required
     description: optional
     sampling_frequency: optional

src/schema/objects/columns.yaml

@@ -183,8 +183,8 @@ hemisphere:
     The hemisphere in which the electrode is placed.
   type: string
   enum:
-    - L
-    - R
+    - $ref: objects.enums.left_hemisphere.value
+    - $ref: objects.enums.right_hemisphere.value
 high_cutoff:
   name: high_cutoff
   display_name: High cutoff
@@ -603,159 +603,62 @@ trigger:
     continuous measurement of the scanner trigger signal
   type: number
 # type column in channels.tsv files
-type__eeg_channels:
-  name: type
-  display_name: Channel type
-  description: |
-    Type of channel; MUST use the channel types listed below.
-    Note that the type MUST be in upper-case.
-  type: string
-  enum:
-    - AUDIO
-    - EEG
-    - EOG
-    - ECG
-    - EMG
-    - EYEGAZE
-    - GSR
-    - HEOG
-    - MISC
-    - PPG
-    - PUPIL
-    - REF
-    - RESP
-    - SYSCLOCK
-    - TEMP
-    - TRIG
-    - VEOG
-type__meg_channels:
-  name: type
-  display_name: Channel type
-  description: |
-    Type of channel; MUST use the channel types listed below.
-    Note that the type MUST be in upper-case.
-  type: string
-  enum:
-    - MEGMAG
-    - MEGGRADAXIAL
-    - MEGGRADPLANAR
-    - MEGREFMAG
-    - MEGREFGRADAXIAL
-    - MEGREFGRADPLANAR
-    - MEGOTHER
-    - EEG
-    - ECOG
-    - SEEG
-    - DBS
-    - VEOG
-    - HEOG
-    - EOG
-    - ECG
-    - EMG
-    - TRIG
-    - AUDIO
-    - PD
-    - EYEGAZE
-    - PUPIL
-    - MISC
-    - SYSCLOCK
-    - ADC
-    - DAC
-    - HLU
-    - FITERR
-    - OTHER
-type__ieeg_channels:
-  name: type
-  display_name: Channel type
-  description: |
-    Type of channel; MUST use the channel types listed below.
-    Note that the type MUST be in upper-case.
-  type: string
-  enum:
-    - EEG
-    - ECOG
-    - SEEG
-    - DBS
-    - VEOG
-    - HEOG
-    - EOG
-    - ECG
-    - EMG
-    - TRIG
-    - AUDIO
-    - PD
-    - EYEGAZE
-    - PUPIL
-    - MISC
-    - SYSCLOCK
-    - ADC
-    - DAC
-    - REF
-    - OTHER
-type__nirs_channels:
-  name: type
-  display_name: Channel type
-  description: |
-    Type of channel; MUST use the channel types listed below.
-    Note that the type MUST be in upper-case.
-  type: string
-  enum:
-    - NIRSCWAMPLITUDE
-    - NIRSCWFLUORESCENSEAMPLITUDE
-    - NIRSCWOPTICALDENSITY
-    - NIRSCWHBO
-    - NIRSCWHBR
-    - NIRSCWMUA
-    - MEGMAG
-    - MEGGRADAXIAL
-    - MEGGRADPLANAR
-    - MEGREFMAG
-    - MEGREFGRADAXIAL
-    - MEGREFGRADPLANAR
-    - MEGOTHER
-    - EEG
-    - ECOG
-    - SEEG
-    - DBS
-    - VEOG
-    - HEOG
-    - EOG
-    - ECG
-    - EMG
-    - TRIG
-    - AUDIO
-    - PD
-    - EYEGAZE
-    - PUPIL
-    - MISC
-    - SYSCLOCK
-    - ADC
-    - DAC
-    - HLU
-    - FITERR
-    - ACCEL
-    - GYRO
-    - MAGN
-    - MISC
-    - OTHER
-type__motion_channels:
+type__channels:
   name: type
   display_name: Channel type
   description: |
     Type of channel; MUST use the channel types listed below.
     Note that the type MUST be in upper-case.
   type: string
   enum:
-    - ACCEL
-    - ANGACCEL
-    - GYRO
-    - JNTANG
-    - LATENCY
-    - MAGN
-    - MISC
-    - ORNT
-    - POS
-    - VEL
+    - $ref: objects.enums.ACCEL.value
+    - $ref: objects.enums.ADC.value
+    - $ref: objects.enums.ANGACCEL.value
+    - $ref: objects.enums.AUDIO.value
+    - $ref: objects.enums.DAC.value
+    - $ref: objects.enums.DBS.value
+    - $ref: objects.enums.ECG.value
+    - $ref: objects.enums.ECOG.value
+    - $ref: objects.enums.EEG.value
+    - $ref: objects.enums.EMG.value
+    - $ref: objects.enums.EOG.value
+    - $ref: objects.enums.EYEGAZE.value
+    - $ref: objects.enums.FITERR.value
+    - $ref: objects.enums.GSR.value
+    - $ref: objects.enums.GYRO.value
+    - $ref: objects.enums.HEOG.value
+    - $ref: objects.enums.HLU.value
+    - $ref: objects.enums.JNTANG.value
+    - $ref: objects.enums.LATENCY.value
+    - $ref: objects.enums.MAGN.value
+    - $ref: objects.enums.MEGGRADAXIAL.value
+    - $ref: objects.enums.MEGGRADPLANAR.value
+    - $ref: objects.enums.MEGMAG.value
+    - $ref: objects.enums.MEGOTHER.value
+    - $ref: objects.enums.MEGREFGRADAXIAL.value
+    - $ref: objects.enums.MEGREFGRADPLANAR.value
+    - $ref: objects.enums.MEGREFMAG.value
+    - $ref: objects.enums.MISC.value
+    - $ref: objects.enums.NIRSCWAMPLITUDE.value
+    - $ref: objects.enums.NIRSCWFLUORESCENSEAMPLITUDE.value
+    - $ref: objects.enums.NIRSCWHBO.value
+    - $ref: objects.enums.NIRSCWHBR.value
+    - $ref: objects.enums.NIRSCWMUA.value
+    - $ref: objects.enums.NIRSCWOPTICALDENSITY.value
+    - $ref: objects.enums.ORNT.value
+    - $ref: objects.enums.OTHER.value
+    - $ref: objects.enums.PD.value
+    - $ref: objects.enums.POS.value
+    - $ref: objects.enums.PPG.value
+    - $ref: objects.enums.PUPIL.value
+    - $ref: objects.enums.REF.value
+    - $ref: objects.enums.RESP.value
+    - $ref: objects.enums.SEEG.value
+    - $ref: objects.enums.SYSCLOCK.value
+    - $ref: objects.enums.TEMP.value
+    - $ref: objects.enums.TRIG.value
+    - $ref: objects.enums.VEL.value
+    - $ref: objects.enums.VEOG.value
 # type column for electrodes.tsv files
 type__electrodes:
   name: type

src/schema/objects/entities.yaml

@@ -132,8 +132,8 @@ hemisphere:
   type: string
   format: label
   enum:
-    - 'L'
-    - 'R'
+    - $ref: objects.enums.left_hemisphere.value
+    - $ref: objects.enums.right_hemisphere.value
 inversion:
   name: inv
   display_name: Inversion Time
@@ -184,8 +184,8 @@ mtransfer:
   type: string
   format: label
   enum:
-    - 'on'
-    - 'off'
+    - $ref: objects.enums.on__mtransfer.value
+    - $ref: objects.enums.off__mtransfer.value
 part:
   name: part
   display_name: Part
@@ -207,10 +207,10 @@ part:
   type: string
   format: label
   enum:
-    - mag
-    - phase
-    - real
-    - imag
+    - $ref: objects.enums.magnitude.value
+    - $ref: objects.enums.phase.value
+    - $ref: objects.enums.real.value
+    - $ref: objects.enums.imaginary.value
 processing:
   name: proc
   display_name: Processed (on device)