Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] microelectrode electrophysiology specification (BEP032) #1705

Open
wants to merge 30 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7ec5d5e
Add first BEP032 objects
JuliaSprenger Nov 9, 2022
2f393ca
Add ephys modality, start adding _channels but ran into a question on…
yarikoptic Feb 14, 2024
648781a
Add _acq- to ephys, add _channels and _coords
yarikoptic Mar 6, 2024
d3c9996
Draft for probes TSV file
TheChymera Mar 6, 2024
08c92cd
Fix preferably typo
yarikoptic Mar 15, 2024
e2030ac
Comment out not yet defined iephys and add ephys to datatypes where i…
TheChymera Apr 17, 2024
53f1087
Added probes file
TheChymera Apr 17, 2024
fa648fe
Describe modality/datatype as "Microelectrode Electrophysiology"
yarikoptic Apr 18, 2024
eef67df
Remove iephys entirely since I think it was just inspired by DANDI la…
yarikoptic Apr 18, 2024
27547f9
Initial content for ephys document and initial filename template tree
yarikoptic Apr 18, 2024
a6ae5e2
Merge remote-tracking branch 'upstream/master' into bep032
Remi-Gau Apr 19, 2024
9a20e4b
pacify pre-commit
Remi-Gau Apr 19, 2024
7609276
add to TOC
Remi-Gau Apr 19, 2024
9be41f9
semantic line break and fix
Remi-Gau Apr 19, 2024
983975a
Addressed minor left over lint complains
yarikoptic Apr 19, 2024
1afdc40
Add "Microelectrode" specialization for Electrophysiology in the cont…
yarikoptic Apr 19, 2024
6b39d64
Fix up for bad latin and adjust wording a bit
yarikoptic Apr 19, 2024
f08b163
Replace fancy unicode “ with simple " .
yarikoptic Apr 19, 2024
73012b3
Use redirected to www.datalad.org and www.nwb.org website URLs and fe…
yarikoptic Apr 19, 2024
a46e439
BEP032: ephys -> microephys with {icephys, ecephys} modalities/suffix…
yarikoptic Jun 5, 2024
05e9611
Clarify wording and do mention `microephys` as shorthand for Microele…
yarikoptic Oct 16, 2024
7ddeb75
Adding markdown portion of the Bep032 (#1960)
Peyman-N Nov 20, 2024
2b9e757
[ENH] Schema changes related to BEP032 (#1826)
ree-gupta Nov 20, 2024
6b11f54
Merge remote-tracking branch 'origin/master' into bep032
yarikoptic Nov 20, 2024
1abebea
Some fixes of formatting and URLs
yarikoptic Dec 17, 2024
2a3757d
Merge remote-tracking branch 'origin/master' into bep032
yarikoptic Dec 18, 2024
3f4e941
Adjusted some formatting and some wording at the beginning of the sec…
yarikoptic Dec 18, 2024
0853f8a
Few fixes to the schema from review
yarikoptic Dec 18, 2024
800ba58
Adopting the markdown line lengths for BEP032 (#2012)
Peyman-N Dec 18, 2024
46c91b0
Fix tabulation in a .tsv + convert to use yaw, pitch, roll
yarikoptic Dec 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ nav:
- Microscopy: modality-specific-files/microscopy.md
- Near-Infrared Spectroscopy: modality-specific-files/near-infrared-spectroscopy.md
- Motion: modality-specific-files/motion.md
- Microelectrode Electrophysiology: modality-specific-files/microelectrode-electrophysiology.md
- Derivatives:
- BIDS Derivatives: derivatives/introduction.md
- Common data types and metadata: derivatives/common-data-types.md
Expand Down
58 changes: 58 additions & 0 deletions src/modality-specific-files/microelectrode-electrophysiology.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Microelectrode Electrophysiology

Support for Microelectrode Electrophysiology was developed as a [BIDS Extension Proposal](../extensions.md#bids-extension-proposals) [BEP032: Animal electrophysiology (ephys)](https://bids.neuroimaging.io/bep032).
yarikoptic marked this conversation as resolved.
Show resolved Hide resolved

Please see [Citing BIDS](../introduction.md#citing-bids) on how to appropriately credit this extension
when referring to it in the context of the academic literature.

This BEP has been initiated by members of the INCF Working Group on Standardized Data Structures,
that was initiated in 2020 to develop a set of specifications and tools
that would allow the standardization of a directory structure for experimental data recorded
with animal models in neuroscience, and its associated metadata.

Please consider joining this working group if you would like to contribute to this effort.
You can also reach the moderators of this BEP through our [main discussion forum](https://github.com/INCF/neuroscience-data-structure/issues), where you can participate in existing discussions or raise new questions / issues.

Most core principles of the original BIDS and particulars of BIDS-iEEG specification are adopted
for this modality as well, though some special considerations and additional fields were added.

Several [example Microelectrode Electrophysiology datasets](https://bids-standard.github.io/bids-examples/#microephys)
have been formatted using this specification and can be used for practical guidance when curating a new dataset.

## Primary data file formats

Unprocessed microelectrode electrophysiology (`microephys`) modality data (of `icephys` or `ecephys` datatypes) must be stored in an [open file format](https://en.wikipedia.org/wiki/Open_format),
while the native format, if different, can be stored in an optional `sourcedata/` directory.
The native file format is used in case conversion elicits the loss of crucial metadata specific to manufacturers and specific acquisition systems.
Metadata should be included alongside the data in the `.json` and `.tsv` files.
The current list of allowed data file formats:

| **Format** | **Extension(s)** | **Description** |
--------------------------------------------------------------------------------------|------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Neuroscience Information Exchange Format](https://nixio.readthedocs.io/en/latest/) | `.nix` | A generic and open framework with an hdf5 backend and a defined interface to many ephys formats via the [Neo library](https://neo.readthedocs.io/en/latest/). The `.nix` file has to contain a valid Neo structure. |
| [Neurodata Without Borders](https://www.nwb.org) | `.nwb` | BRAIN Initiative Data Standard based on an hdf5 backend ... |
yarikoptic marked this conversation as resolved.
Show resolved Hide resolved

Both of these formats can also store essential metadata of the datasets.
Some of these need to be duplicated in BIDS `.tsv` and `.json` sidecar files.
Even though the duplication requires additional effort to ensure the consistency of the data, it provides a number of advantages:
- Making the dataset easier for humans to scan as essential information is easily accessible without loading the data files
- The dataset follows the BIDS standard and can benefit from tools building on top of this standard, starting with [bids-validator](https://github.com/bids-standard/bids-validator).
- It simplifies the separation of data and basic metadata, for example to publish a dataset in a light-weight fashion with access to the data files on request (as implemented by [DataLad](https://www.datalad.org)).

##

<!--
This block generates a filename templates.
The inputs for this macro can be found in the directory
src/schema/rules/files/raw
and a guide for using macros can be found at
https://github.com/bids-standard/bids-specification/blob/master/macros_doc.md
-->
{{ MACROS___make_filename_template(
"raw",
datatypes=["ecephys", "icephys"],
suffixes=["ecephys", "icephys", "events"]
)
}}

<!-- Link Definitions -->
93 changes: 93 additions & 0 deletions src/schema/objects/columns.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,41 @@ age:
for privacy purposes.
type: number
unit: year
alpha_rotation:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really recommend renaming the rotation axes to yaw, roll, and pitch (that would be the analogous angle order). There was no consensus either way on the google docs discussion. Someone said both are confusing, which I guess might be expected, but alpha, beta, gamma, are just more confusing...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree -- very adhoc. Let's discuss in google doc

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Could you chime in? I think the other guy commenting might be amenable to accepting this as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that both are confusing, but at least alpha, beta and gamma are SO confusing that everyone realizes that additional specification is needed to define them properly. With roll, yaw and pitch it seems at first that all is clear, until you have a number of different people go through different use cases. See the more challenging examples that I posted on the google doc under https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Copy link
Collaborator

@TheChymera TheChymera Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robertoostenveld sorry, I only now saw the update, reply there. I don't think confusion on purpose is good in this case, because the documentation is very eclectic so we might be sending people down rabbit holes. Wiki, where people will invariably go first, does a particularly poor job explaining both euler and TB angles for the casual non-mathematics-versed user. The only thing that wiki has going for it here are the aircraft animations on the TB page. Yaw, Pitch, Roll, will be intuited correctly as long as we specify the starting postion. That we can do (1) as (I think, it's pretty vague) is currently proposed, aligning the implant with the world coordinate system (meaning most implants will be at yaw 0 pitch -90 roll 0) or (2) relative to the implantation stereotax normal (meaning most implants will be at 0 0 0).

For comparison, Euler commonly has the normal pointing up so most implants will be.... 0 -180 0 🤔

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the last BEP032 meeting we discussed further and wanted to follow this approach now (based on the Allen Inst. standard and IBL standard):

Assumption: x,y,z is posterior, ventral, right (unit needs to be specified).

Translational origin (0,0,0) needs to be defined (typically Bregma for rodents).

Rotational origin (0,0,0) is the probe facing up with the tip facing forward. Rotations are all clockwise in degrees and around the tip. For multi-shank probes, the “tip” of the probe is defined as the end of the left shank if you are looking at the electrodes.

  • yaw: clockwise when looking down
  • pitch: In the direction of the electrode face
  • roll: clockwise when looking down at the probe

The depth (unit needs to be specified) is a translation in the direction that the tip is pointing.

We need to add a “probe_model”, which references probeinterface_library

  • versioning of probe library files (raised here)
  • annotation of “tip” position (raised here)

NOTE: We need to change the electrode x,y,z.
X,y,z in BIDS refers to location in brain, not on probe.

yarikoptic marked this conversation as resolved.
Show resolved Hide resolved
name: alpha_rotation
display_name: Alpha rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
yarikoptic marked this conversation as resolved.
Show resolved Hide resolved
unit:
degree
associated_brain_region:
name: associated_brain_region
display_name: Associated brain region
description: |
A textual indication on the location of the probe,
preferably species-independent terms as obtained, for example from Uberon.
type: string
associated_brain_region_id:
name: associated_uberon_brain_region_id
display_name: Associated brain region identifier
description: |
An identifier of the associated brain region based on the Uberon ontology
for anatomical structures in animals, for example "UBERON:0010415"
type: number
associated_brain_region_quality_type:
name: associated_brain_region_quality_type
display_name: Associated brain region quality type
description: |
The method used to identify the associated brain region (estimated|proof)
depending on anatomical pictures proofing the location or indirect estimation of the location.
type: string
beta_rotation:
name: beta_rotation
display_name: Beta rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
yarikoptic marked this conversation as resolved.
Show resolved Hide resolved
unit:
degree
cardiac:
name: cardiac
display_name: Cardiac measurement
Expand Down Expand Up @@ -84,6 +119,19 @@ component:
- $ref: objects.enums.quat_z.value
- $ref: objects.enums.quat_w.value
- n/a
contact_count:
name: contact_count
display_name: Contact count
description: |
Number of miscellaneous analog contacts for auxiliary signals.
type: number
depth__probes:
name: height
display_name: Height
description: |
Physical height of the probe.
type: number
unit: mm
detector__channels:
name: detector
display_name: Detector Name
Expand Down Expand Up @@ -131,6 +179,12 @@ description:
description: |
Brief free-text description of the channel, or other information of interest.
type: string
device_serial_number:
name: device_serial_number
display_name: Device serial number
description: |
The serial number of the probe (provided by the manufacturer).
type: string
description__optode:
name: description
display_name: Description
Expand Down Expand Up @@ -172,6 +226,13 @@ filename:
Relative paths to files.
type: string
format: participant_relative
gamma_rotation:
name: gamma_rotation
display_name: Gamma rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
yarikoptic marked this conversation as resolved.
Show resolved Hide resolved
unit:
degree
group__channel:
name: group
display_name: Channel group
Expand Down Expand Up @@ -213,6 +274,13 @@ handedness:
- AMBIDEXTROUS
- Ambidextrous
- n/a
height__probes:
name: height
display_name: Height
description: |
Physical height of the probe.
type: number
unit: mm
hemisphere:
name: hemisphere
display_name: Electrode hemisphere
Expand Down Expand Up @@ -288,6 +356,12 @@ material:
description: |
Material of the electrode (for example, `Tin`, `Ag/AgCl`, `Gold`).
type: string
material__probes:
name: material__probes
display_name: Material
description: |
A textual description of the base material of the probe.
type: string
metabolite_parent_fraction:
name: metabolite_parent_fraction
display_name: Metabolite parent fraction
Expand Down Expand Up @@ -412,6 +486,12 @@ reference__ieeg:
- type: string
enum:
- n/a
reference_atlas:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The “reference atlas” if you visually verify the area is basically the atlas you looked at before you nod and said “eh, good enough”. This seems at once an overly detailed (does this really matter? the coordinates are commonly given with sub-mm precision) and underdetermined (what coordinates did you look at in the atlas?). I think it's best just to drop this.

name: reference_atlas
display_name: Reference atlas
description: |
Name of reference atlas used for associated brain region identification, preferably an ebrains supported atlas.
type: string
reference_frame:
name: reference_frame
display_name: Reference frame
Expand Down Expand Up @@ -724,6 +804,12 @@ type__optodes:
- $ref: objects.enums.source.value
- $ref: objects.enums.detector.value
- n/a
type__probes:
name: type
display_name: Type
description: |
The type of the probe.
type: string
units:
name: units
display_name: Units
Expand Down Expand Up @@ -806,6 +892,13 @@ whole_blood_radioactivity:
Radioactivity in whole blood samples,
in unit of radioactivity measurements in whole blood samples (for example, `kBq/mL`).
type: number
width__probes:
name: width
display_name: Width
description: |
Physical width of the probe.
type: number
unit: mm
x:
name: x
display_name: X position
Expand Down
10 changes: 10 additions & 0 deletions src/schema/objects/datatypes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ dwi:
display_name: Diffusion-Weighted Imaging
description: |
Diffusion-weighted imaging (DWI).
ecephys:
value: ecephys
display_name: Extracellular Electrophysiology
description: |
Electrophysiological data from extracellular microelectrodes
eeg:
value: eeg
display_name: Electroencephalography
Expand All @@ -30,6 +35,11 @@ func:
display_name: Task-Based Magnetic Resonance Imaging
description: |
Task (including resting state) imaging data
icephys:
value: icephys
display_name: Intracellular Electrophysiology
description: |
Electrophysiological data acquired from intracellular or patch microelectrodes
ieeg:
value: ieeg
display_name: Intracranial electroencephalography
Expand Down
6 changes: 3 additions & 3 deletions src/schema/objects/entities.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -331,12 +331,12 @@ split:
display_name: Split
description: |
In the case of long data recordings that exceed a file size of 2Gb,
`.fif` files are conventionally split into multiple parts.
Each of these files has an internal pointer to the next file.
`.fif`, `.nwb`, `.nix` files are conventionally split into multiple parts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NWB files that exceed 2GB are not "conventionally" split into multiple parts. I don't know if this is true for NIX but I doubt it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each of the `.fif` files has an internal pointer to the next file.
This is important when renaming these split recordings to the BIDS convention.

Instead of a simple renaming, files should be read in and saved under their
new names with dedicated tools like [MNE-Python](https://mne.tools/),
new names with dedicated tools like [MNE-Python](https://mne.tools/) for `.fif`,
which will ensure that not only the filenames, but also the internal file pointers, will be updated.

It is RECOMMENDED that `.fif` files with multiple parts use the `split-<index>` entity to indicate each part.
Expand Down
5 changes: 5 additions & 0 deletions src/schema/objects/extensions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,11 @@ niigz:
display_name: Compressed NIfTI
description: |
A compressed Neuroimaging Informatics Technology Initiative (NIfTI) data file.
nix:
value: .nix
display_name: Neuroscience Information Exchange Format
description: |
A [Neuroscience Information Exchange](https://nixio.readthedocs.io) file.
nwb:
value: .nwb
display_name: Neurodata Without Borders Format
Expand Down
3 changes: 3 additions & 0 deletions src/schema/objects/modalities.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,6 @@ motion:
nirs:
display_name: Near-Infrared Spectroscopy
description: Data acquired with NIRS.
microephys:
display_name: Microelectrode Electrophysiology
description: Electrophysiological data acquired using microelectodes.
20 changes: 20 additions & 0 deletions src/schema/objects/suffixes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,11 @@ dwi:
display_name: Diffusion-weighted image
description: |
Diffusion-weighted imaging contrast (specialized T2 weighting).
ecephys:
value: ecephys
display_name: Extracellular Electrophysiology
description: |
Extracellular electrophysiological data.
eeg:
value: eeg
display_name: Electroencephalography
Expand Down Expand Up @@ -612,6 +617,11 @@ hipCT:
display_name: HiP-CT
description: |
Hierarchical Phase-Contrast Tomography imaging data
icephys:
value: icephys
display_name: Inracellular Electrophysiology
description: |
Intracellular electrophysiological data.
ieeg:
value: ieeg
display_name: Intracranial Electroencephalography
Expand Down Expand Up @@ -744,6 +754,16 @@ physio:
display_name: Physiological recording
description: |
Physiological recordings such as cardiac and respiratory signals.
probe:
value: probe
display_name: A recording probe
description: |
A probe with one or more recording contacts.
probes:
value: probes
display_name: Implanted physical devices
description: |
Physical devices used for recording microelectrode electrophysiology data, whether chronically or acutely implanted.
probseg:
value: probseg
display_name: Probabilistic Segmentation
Expand Down
21 changes: 21 additions & 0 deletions src/schema/rules/files/raw/channels.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ channels:
acquisition: optional
run: optional

# microephys has an additional 'sample' entity
channels__microephys:
$ref: rules.files.raw.channels.channels
datatypes:
- icephys
- ecephys
entities:
$ref: rules.files.raw.channels.channels.entities
sample: optional

yarikoptic marked this conversation as resolved.
Show resolved Hide resolved
# MEG has an additional entity available
channels__meg:
$ref: rules.files.raw.channels.channels
Expand Down Expand Up @@ -57,6 +67,17 @@ coordsystem__eeg:
$ref: rules.files.raw.channels.coordsystem.entities
space: optional

# microephys has sample AND space entities
coordsystem__microephys:
$ref: rules.files.raw.channels.coordsystem
datatypes:
- ecephys
- icephys
entities:
$ref: rules.files.raw.channels.coordsystem.entities
sample: optional
space: optional

electrodes:
suffixes:
- electrodes
Expand Down
38 changes: 38 additions & 0 deletions src/schema/rules/files/raw/microephys.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
microephys:
suffixes:
- ecephys
- icephys
extensions:
- .nwb
# possible future: serialization in .zarr format to accompany .ome.zarr
# - .nwb.zarr
- .nix
datatypes:
- ecephys
- icephys
entities:
subject: required
session: optional
sample: optional
task: optional
acquisition: optional
run: optional
split: optional

probes:
suffixes:
- probes
extensions:
- .tsv
datatypes:
- ecephys
- icephys
entities:
subject: required
session: optional
sample: optional
task: optional
acquisition: optional
run: optional
split: optional