Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Added the specification for using HED libraries in BIDS #1106

Merged
merged 18 commits into from
Aug 31, 2022
+74 −8
Merged

[ENH] Added the specification for using HED libraries in BIDS #1106

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
6ce58dc
Added the specification for using HED libraries in BIDS
VisLab May 23, 2022
976559a
Added newline at end of hed appendices
VisLab May 23, 2022
6555ad5
Corrected the metadata.yml spacing
VisLab May 23, 2022
dc723e4
Added requirement that user-chosen prefixes be alphanumeric
VisLab May 24, 2022
d52c011
Merge branch 'master' of https://github.com/bids-standard/bids-specif…
VisLab Jun 9, 2022
8039ec9
Simplified HEDVersion to be array rather than object
VisLab Jun 9, 2022
ccf749b
Provided example using only SCORE library
VisLab Jun 9, 2022
ce26293
Made some typo and editorial corrections
VisLab Jun 9, 2022
a0a7b36
Changed base schema to standard schema
VisLab Jun 9, 2022
37e9b8f
Removed extra colon
VisLab Jun 9, 2022
42708ef
Update src/99-appendices/03-hed.md
VisLab Jun 11, 2022
f26bcd1
Update src/99-appendices/03-hed.md
VisLab Jun 11, 2022
cfea5f7
Merge branch 'master' into hed-library
sappelhoff Aug 29, 2022
5310be6
Update src/99-appendices/03-hed.md
sappelhoff Aug 29, 2022
a49df64
Update metadata.yaml
sappelhoff Aug 29, 2022
55d79a6
add new format: hed_version
sappelhoff Aug 30, 2022
5c7045d
line break pattern
sappelhoff Aug 30, 2022
6101459
Update src/schema/objects/metadata.yaml
sappelhoff Aug 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 59 additions & 7 deletions src/99-appendices/03-hed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,21 +167,73 @@ This allows for a proper validation of the HED annotations
(for example using the `bids-validator`).

Example: The following `dataset_description.json` file specifies that the
[`HED8.0.0.xml`](https://github.com/hed-standard/hed-specification/tree/master/hedxml/HED8.0.0.xml)
[`HED8.1.0.xml`](https://github.com/hed-standard/hed-specification/tree/master/hedxml/HED8.1.0.xml)
file from the `hedxml` directory of the
[`hed-specification`](https://github.com/hed-standard/hed-specification)
repository on GitHub should be used to validate the study event annotations.

```JSON
{
"Name": "A great experiment",
"BIDSVersion": "1.6.0",
"HEDVersion": "8.0.0"
"BIDSVersion": "1.7.0",
sappelhoff marked this conversation as resolved.
Show resolved Hide resolved
"HEDVersion": "8.1.0"
}
```

If you omit the `HEDVersion` field from the dataset description file,
any present HED information will be validated using the latest version of the HED schema,
which is bound to result in problems.
Hence, it is strongly RECOMMENDED that the `HEDVersion` field be included when using HED
in a BIDS dataset.
any present HED information will be validated using the latest version of the HED schema.
This is bound to result in problems, and hence, it is strongly RECOMMENDED that the
`HEDVersion` field be included when using HED in a BIDS dataset.

### Using HED library schemas

HED also allows you to use one or more specialized vocabularies along with or instead of
the standard vocabulary. These specialized vocabularies are developed by
communities of users and are available in the GitHub
[hed-schema-library](https://github.com/hed-standard/hed-schema-library) repository.
Library schema are specified in the form `<library-name<_>library-version>`.

Example: The following `dataset_description.json` file specifies that the
[HED8.1.0.xml](https://github.com/hed-standard/hed-specification/tree/master/hedxml/HED8.1.0.xml)
standard schema should be used along with the
SCORE library for clinical neurological annotation and a test library.
These later schemas are located at
[HED_score_0.0.1.xml](https://github.com/hed-standard/hed-schema-library/blob/main/library_schemas/score/hedxml/HED_score_0.0.1.xml) and
[HED_testlib_1.0.2.xml](https://github.com/hed-standard/hed-schema-library/blob/main/library_schemas/testlib/hedxml/HED_testlib_1.0.2.xml), respectively.

```JSON
{
"Name": "A great experiment",
"BIDSVersion": "1.7.0",
"HEDVersion": ["8.1.0", "sc:score_0.0.1", "ts:testlib_1.0.2"]
}
```
The `sc:` and `ts:` are user-chosen prefixes used to distinguish the sources
of the terms in the HED annotation.
These prefixes must be alphanumeric.
yarikoptic marked this conversation as resolved.
Show resolved Hide resolved
sappelhoff marked this conversation as resolved.
Show resolved Hide resolved

The following HED annotation from this dataset uses the `sc:` prefix with
`Photomyogenic-response` and `Wicket-spikes` because these terms are from the
SCORE library, while `Data-feature` is from the standard HED schema.

```Text
Data-feature, sc:Photomyogenic-response, sc:Wicket-spikes
```

If only one schema is being used for annotation, the prefix can be omitted entirely.
The following `dataset_description.json` indicates that only the SCORE library version
0.0.1 will be used for HED annotation in this dataset.
sappelhoff marked this conversation as resolved.
Show resolved Hide resolved

```JSON
{
"Name": "A great experiment",
"BIDSVersion": "1.7.0",
"HEDVersion": "score_0.0.1"
}
```

The corresponding notations in the dataset do not have a prefix:

```Text
Photomyogenic-response, Wicket-spikes
```
7 changes: 6 additions & 1 deletion src/schema/objects/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1022,7 +1022,12 @@ HEDVersion:
description: |
If HED tags are used:
The version of the HED schema used to validate HED tags for study.
type: string
May include a single schema or a base schema and one or more library schema.
anyOf:
- type: string
- type: array
items:
type: string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be beneficial to think about what validation we can build into the schema here apart from "string". E.g., checking for permissible versioning formats, occurrence of : and _ characters, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good idea if someone knows how to do it that would be great.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try adding this behemoth regex to src/schema/objects/formats.yaml (the first two groups are for the library nickname and library name, and the rest was adapted from https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string):

^(?:[a-zA-Z]+:)?(?:[a-zA-Z]+_)?(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)(?:-(?:(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?:[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @happy5214 what's your opinion on adding this regex for validation? Will it cause more pain than it's worth, or should we do it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was pinged, but I don't know if the question was directed toward me or if you were just thanking me. The tail end of the regex could be simplified to the following if we were to ban the use of pre-release schema versions:

^(?:[a-zA-Z]+:)?(?:[a-zA-Z]+_)?(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)$

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both to thank you and to get your opinion, because you called it a "behemoth" and I wasn't sure whether that means you advise against adding this. :-)

Meanwhile this discussion progresses in the main thread

Haematocrit:
name: Haematocrit
description: |
Expand Down