Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Hierarchical Event Descriptors (HED) page update #1623

Merged
merged 2 commits into from
Oct 2, 2023
Merged

[FIX] Hierarchical Event Descriptors (HED) page update #1623

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
158c8a4
[FIX] Hierarchical Event Descriptors (HED) page update
tpatpa Sep 29, 2023
518d88d
FIX: Implement Kay's suggestions
effigies Oct 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 93 additions & 46 deletions src/appendices/hed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,19 @@ A HED annotation consists of terms selected from a controlled
hierarchical vocabulary (the HED schema).
Individual terms are comma-separated and may be grouped using parentheses to indicate
association.
See [https://www.hedtags.org/display_hed.html](https://www.hedtags.org/display_hed.html)
See the [HED Schema Browser](https://www.hedtags.org/display_hed.html)
to view the HED schema and the
[HED resources](https://www.hed-resources.org/en/latest/) site for additional information.

Starting with HED version 8.0.0, HED allows users to annotate using individual
terms or partial paths in the HED vocabulary (for example `Red` or `Visual-presentation`)
terms or partial paths in the HED vocabulary (for example, `Red` or `Visual-presentation`)
rather than the full paths in the HED hierarchy (
`Property/Sensory-property/Sensory-attribute/Visual-attribute/Color/CSS-color/Red-color/Red`
or
`Property/Sensory-property/Sensory-presentation/Visual-presentation`).

HED specific tools MUST treat the short (single term) and long (full path) HED tag forms interchangeably,
converting between the forms when necessary based on the HED schema.
converting between the forms, when necessary, based on the HED schema.
Examples of test datasets using the various forms can be found in
[hed-examples/datasets](https://github.com/hed-standard/hed-examples/tree/main/datasets)
on GitHub.
Expand All @@ -40,14 +40,14 @@ files in various places in the dataset hierarchy
Dataset curators MAY also include additional columns and define their
meanings in associated JSON sidecar files (`events.json`).

Example: An excerpt from an `events.tsv` file containing three columns
**Example:** An excerpt from an `events.tsv` file containing three columns
(`trial_type`, `response_time`, and `stim_file`) in addition to
the required `onset` and `duration` columns.

```Text
onset duration trial_type response_time stim_file
1.2 0.6 go 1.435 images/red_square.jpg
5.6 0.6 stop 1.739 images/blue_square.jpg
5.6 0.6 stop n/a images/blue_square.jpg
```

The `trial_type` column in the above example contains a limited number of distinct
Expand All @@ -66,13 +66,13 @@ The HED annotation for a value column must include a `#` placeholder,
which dedicated HED tools MUST replace by the actual column value when the annotations
are assembled for analysis.

Example: An accompanying `events.json` sidecar describing both categorical and
**Example:** An accompanying `events.json` sidecar describing both categorical and
value columns of the previous example.
The `duration` column is also annotated as a value column.

```JSON
{
"Duration": {
"duration": {
"LongName": "Image duration",
"Description": "Duration of the image presentations",
"Units": "s",
Expand All @@ -82,12 +82,12 @@ The `duration` column is also annotated as a value column.
"LongName": "Event category",
"Description": "Indicator of type of action that is expected",
"Levels": {
"go": "A blue square is displayed to indicate starting",
"stop": "A red square is displayed to indicate stopping"
"go": "A red square is displayed to indicate starting",
"stop": "A blue square is displayed to indicate stopping"
},
"HED": {
"go": "Sensory-event, Visual-presentation, (Square, Blue)",
"stop": "Sensory-event, Visual-presentation, (Square, Red)"
"go": "Sensory-event, Visual-presentation, (Square, Red)",
"stop": "Sensory-event, Visual-presentation, (Square, Blue)"
}
},
"response_time": {
Expand All @@ -106,18 +106,20 @@ The `duration` column is also annotated as a value column.

Dedicated HED tools MUST assemble the HED annotation for each event (row) by concatenating the
annotations for each column, along with the annotation contained directly in a `HED` column
of that row as described in the next section.
of that row, as described in the next section.

Example: The fully assembled annotation for the first event in the above
**Example:** The fully assembled annotation for the first event in the above
`events.tsv` file with onset `1.2` (the first row) is:

```Text
Duration/0.6 s, Sensory-event, Visual-presentation, (Square, Blue),
(Delay/1.435 ms, Agent-action, (Experiment-participant, (Press, Mouse-button))),
Duration/0.6 s, Sensory-event, Visual-presentation,
((Square, Red), (Computer-screen, Center-of)),
(Delay/1.435 ms, Agent-action, (Experiment-participant,
(Press, Mouse-button))),
Pathname/images/red_square.jpg
```

## Annotation using the `HED` column
### Annotation using the `HED` column

Another tagging strategy is to annotate individual events directly by
including a `HED` column in the `events.tsv` file.
Expand All @@ -138,7 +140,7 @@ event annotations.

Annotations placed in sidecars are the RECOMMENDED way to annotate data using HED.
These annotations are preferred to those placed
directly in the `HED` column, because they are simpler, more compact,
directly in the `HED` column because they are simpler, more compact,
more easily edited, and less prone to inconsistencies.

## HED and the BIDS inheritance principle
Expand All @@ -162,41 +164,61 @@ The HED vocabulary is specified by a HED schema,
which delineates the allowed HED path strings.
The version of HED used in tagging a dataset should be provided in the `HEDVersion`
field of the `dataset_description.json` file located in the dataset root directory.
This allows for a proper validation of the HED annotations
(for example using the `bids-validator`).
This allows for properly validating the HED annotations
(for example, using the `bids-validator`).

Example: The following `dataset_description.json` file specifies that the
[`HED8.1.0.xml`](https://github.com/hed-standard/hed-schemas/blob/main/standard_schema/hedxml/HED8.1.0.xml)
**Example:** The following `dataset_description.json` file specifies that the
[`HED8.2.0.xml`](https://github.com/hed-standard/hed-schemas/blob/main/standard_schema/hedxml/HED8.2.0.xml)
file from the `standard_schema/hedxml` directory of the
[`hed-schemas`](https://github.com/hed-standard/hed-schemas)
repository on GitHub should be used to validate the study event annotations.

```JSON
{
"Name": "A great experiment",
"BIDSVersion": "1.7.0",
"HEDVersion": "8.1.0"
"BIDSVersion": "1.8.0",
"HEDVersion": "8.2.0"
}
```

If you omit the `HEDVersion` field from the dataset description file,
a warning will be generated and
any present HED information will be validated using the latest version of the HED schema.
This is bound to result in problems, and hence, it is strongly RECOMMENDED that the
`HEDVersion` field be included when using HED in a BIDS dataset.
The BIDS validator will generate an error if your dataset uses HED
and the `HEDVersion` field is missing from the dataset description file.
To avoid this, include a `HEDVersion` field in the `dataset_description.json`
if you are using HED annotations.

### Using HED library schemas

HED also allows you to use one or more specialized vocabularies along with or instead of
the standard vocabulary. These specialized vocabularies are developed by
communities of users and are available in the GitHub
[hed-schemas](https://github.com/hed-standard/hed-schemas) repository.
Library schema are specified in the form `<library-name<_>library-version>`.

Example: The following `dataset_description.json` file specifies that the
HED also allows you to use one or more specialized vocabularies
along with or instead of the standard vocabulary.
These specialized vocabularies are developed by communities of users
and are available in the
[hed-schemas](https://github.com/hed-standard/hed-schemas) GitHub repository.
A library schema is specified in the form `<library-name<_>library-version>`.

#### Partnered library schemas

A partnered schema is one whose vocabulary trees are merged with
its standard schema partner when the schema is released.
Thus, the two vocabularies appear as one vocabulary to the annotator.
Partnered library schemas were introduced in HED specification version 3.2.0
and are supported by HED standard schema versions ≥ 8.2.0.
Each partnered library schema is tied to a specific version of
the HED standard schema as specified in its header.
A given library schema version is either partnered or standalone.

VisLab marked this conversation as resolved.
Show resolved Hide resolved
**Note:** Whether a particular library schema version is partnered or
unpartnered is fixed when the library is released and cannot be changed.
For example,
[HED-SCORE version 1.0.0](https://github.com/hed-standard/hed-schemas/blob/main/library_schemas/score/hedwiki/HED_score_1.0.0.mediawiki)
is unpartnered, but [HED-SCORE version 1.1.0](https://github.com/hed-standard/hed-schemas/blob/main/library_schemas/score/hedwiki/HED_score_1.1.0.mediawiki)
is partnered with standard schema version 8.2.0.

##### Unpartnered library schema example

The following `dataset_description.json` file specifies that the
[HED8.1.0.xml](https://github.com/hed-standard/hed-schemas/blob/main/standard_schema/hedxml/HED8.1.0.xml)
standard schema should be used along with the
SCORE library for clinical neurological annotation located at
standard schema should be used along with the HED-SCORE library schema
for clinical neurological annotation located at
[HED_score_1.0.0.xml](https://github.com/hed-standard/hed-schemas/blob/main/library_schemas/score/hedxml/HED_score_1.0.0.xml).

```JSON
Expand All @@ -206,22 +228,24 @@ SCORE library for clinical neurological annotation located at
"HEDVersion": ["8.1.0", "sc:score_1.0.0"]
}
```
The `sc:` is a user-chosen prefixes used to distinguish the sources
The `sc:` is a user-chosen prefix used to distinguish the source schemas
of the terms in the HED annotation.
The prefixes MUST be alphanumeric.
Any number of prefixed schemas may be used in addition to a non-prefixed one.

The following HED annotation from this dataset uses the `sc:` prefix with
`Photomyogenic-response` and `Wicket-spikes` because these terms are from the
SCORE library, while `Data-feature` is from the standard HED schema.
`Eye-blink-artifact` and `Seizure-PNES` because these terms are from the
HED-SCORE library schema, while `Data-feature` is from the standard HED schema.

```Text
Data-feature, sc:Photomyogenic-response, sc:Wicket-spikes
Data-feature, sc:Eye-blink-artifact, sc:Seizure-PNES
```

If only one schema is being used for annotation, the prefix can be omitted entirely.
The following `dataset_description.json` indicates that only the SCORE library version
0.0.1 will be used for HED annotation in this dataset.
##### Single unpartnered library schema example

If only one schema is used for annotation, the prefix can be omitted entirely.
The following `dataset_description.json` indicates that only the HED-SCORE library schema version
1.0.0 will be used for HED annotation in this dataset.

```JSON
{
Expand All @@ -231,8 +255,31 @@ The following `dataset_description.json` indicates that only the SCORE library v
}
```

The corresponding notations in the dataset do not have a prefix:
The corresponding annotations in the dataset do not have a prefix:

```Text
Eye-blink-artifact, Seizure-PNES
```

##### Partnered library schema example

The following `dataset_description.json` file specifies that
the HED-SCORE library schema
[version 1.1.0](https://github.com/hed-standard/hed-schemas/blob/main/library_schemas/score/hedwiki/HED_score_1.1.0.mediawiki) is used.
This particular library schema version is partnered with the standard schema version
[8.2.0](https://github.com/hed-standard/hed-schemas/blob/main/standard_schema/hedxml/HED8.2.0.xml).

```JSON
{
"Name": "A great experiment",
"BIDSVersion": "1.8.0",
"HEDVersion": "score_1.1.0"
}
```
The corresponding annotations in the dataset use tags from the
HED-SCORE library schema (`Eye-blink-artifact` and `Seizure-PNES`) and from the standard HED (`Data-feature`)
as follows:

```Text
Photomyogenic-response, Wicket-spikes
Data-feature, Eye-blink-artifact, Seizure-PNES
```