Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] microelectrode electrophysiology specification (BEP032) #1705

Open
wants to merge 29 commits into
base: master
Choose a base branch
from

Conversation

yarikoptic
Copy link
Collaborator

@yarikoptic yarikoptic commented Feb 14, 2024

Replaces #1352 submitted from a fork outside of bids-specification.

Add specification for microelectrode electrohpysiology datasets based on the BEP032 proposal

Note

We meet regularly and everyone is welcome

Next meeting: insert date on URL to join

Communication channel: https://framalistes.org/sympa/info/neuroscience-data-structure


TODOs

Issues this PR would likely to address

Issues to see being addressed while working on this BEP (likely to move above) or not (moved below):

Other issues which relate but not in scope here and provided for reference/backreference

@@ -42,6 +42,38 @@ age:
for privacy purposes.
type: number
unit: year
alpha_rotation:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really recommend renaming the rotation axes to yaw, roll, and pitch (that would be the analogous angle order). There was no consensus either way on the google docs discussion. Someone said both are confusing, which I guess might be expected, but alpha, beta, gamma, are just more confusing...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree -- very adhoc. Let's discuss in google doc

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Could you chime in? I think the other guy commenting might be amenable to accepting this as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that both are confusing, but at least alpha, beta and gamma are SO confusing that everyone realizes that additional specification is needed to define them properly. With roll, yaw and pitch it seems at first that all is clear, until you have a number of different people go through different use cases. See the more challenging examples that I posted on the google doc under https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Copy link
Collaborator

@TheChymera TheChymera Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robertoostenveld sorry, I only now saw the update, reply there. I don't think confusion on purpose is good in this case, because the documentation is very eclectic so we might be sending people down rabbit holes. Wiki, where people will invariably go first, does a particularly poor job explaining both euler and TB angles for the casual non-mathematics-versed user. The only thing that wiki has going for it here are the aircraft animations on the TB page. Yaw, Pitch, Roll, will be intuited correctly as long as we specify the starting postion. That we can do (1) as (I think, it's pretty vague) is currently proposed, aligning the implant with the world coordinate system (meaning most implants will be at yaw 0 pitch -90 roll 0) or (2) relative to the implantation stereotax normal (meaning most implants will be at 0 0 0).

For comparison, Euler commonly has the normal pointing up so most implants will be.... 0 -180 0 🤔

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the last BEP032 meeting we discussed further and wanted to follow this approach now (based on the Allen Inst. standard and IBL standard):

Assumption: x,y,z is posterior, ventral, right (unit needs to be specified).

Translational origin (0,0,0) needs to be defined (typically Bregma for rodents).

Rotational origin (0,0,0) is the probe facing up with the tip facing forward. Rotations are all clockwise in degrees and around the tip. For multi-shank probes, the “tip” of the probe is defined as the end of the left shank if you are looking at the electrodes.

  • yaw: clockwise when looking down
  • pitch: In the direction of the electrode face
  • roll: clockwise when looking down at the probe

The depth (unit needs to be specified) is a translation in the direction that the tip is pointing.

We need to add a “probe_model”, which references probeinterface_library

  • versioning of probe library files (raised here)
  • annotation of “tip” position (raised here)

NOTE: We need to change the electrode x,y,z.
X,y,z in BIDS refers to location in brain, not on probe.

@@ -391,6 +462,12 @@ reference__ieeg:
- type: string
enum:
- n/a
reference_atlas:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The “reference atlas” if you visually verify the area is basically the atlas you looked at before you nod and said “eh, good enough”. This seems at once an overly detailed (does this really matter? the coordinates are commonly given with sub-mm precision) and underdetermined (what coordinates did you look at in the atlas?). I think it's best just to drop this.

@TheChymera TheChymera closed this Mar 7, 2024
@TheChymera TheChymera deleted the bep032 branch March 7, 2024 22:13
@TheChymera TheChymera restored the bep032 branch March 7, 2024 22:15
@TheChymera TheChymera reopened this Mar 11, 2024
@TheChymera
Copy link
Collaborator

TheChymera commented Mar 12, 2024

@TheChymera
Copy link
Collaborator

@Remi-Gau Remi-Gau changed the title [ENH] Add BEP032 (microelectrode electrophysiology) specification [ENH] microelectrode electrophysiology specification (BEP032) Apr 18, 2024
Peyman-N and others added 2 commits November 20, 2024 10:11
* test

* MACROS___make_suffix_table

* markdown

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* undoing changes to yaml files

* adding chanell

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update src/modality-specific-files/microelectrode-electrophysiology.md

Co-authored-by: Yaroslav Halchenko <[email protected]>

* Update src/modality-specific-files/microelectrode-electrophysiology.md

Co-authored-by: Yaroslav Halchenko <[email protected]>

* Update src/modality-specific-files/microelectrode-electrophysiology.md

Co-authored-by: Yaroslav Halchenko <[email protected]>

* Update src/modality-specific-files/microelectrode-electrophysiology.md

Co-authored-by: Yaroslav Halchenko <[email protected]>

* Update src/modality-specific-files/microelectrode-electrophysiology.md

Co-authored-by: Yaroslav Halchenko <[email protected]>

* Update src/modality-specific-files/microelectrode-electrophysiology.md

Co-authored-by: Yaroslav Halchenko <[email protected]>

* Update src/modality-specific-files/microelectrode-electrophysiology.md

Co-authored-by: Yaroslav Halchenko <[email protected]>

* adding General ephys metadata

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Examples of real datasets

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor tuneups to formatting

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Yaroslav Halchenko <[email protected]>
* [FIX] Minor YAML formatting

* [ENH] Added additional birthdate column for participants.tsv

* [ENH] Added age_category, age_type and corresponding enums

* [ENH] Added surgery_date column

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reverted changes that better suit in a PR against master

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [ENH] Added tabular_data file and columns for _probes.tsv

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [ENH] Added tabular_data rules and columns for _electrodes.tsv

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [ENH] Added tabular_data rules for _electrodes.tsv

* [ENH] Added tabular_data rules and columns for _channels.tsv

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [FIX] Fixed pre-commit errors

* [ENH] Added additional ephys columns for events.tsv

* [ENH] Added tabular data rules for events.tsv to include ephys related columns

* [ENH] Added metadata entities for ephys setup metadata

* [ENH] Created sidecar rule file for ephys metadata and added data origin and setup fields

* Made schema changes to match microephys data type

* Renamed rule files from ephys to microephys

* Made schema changes to match microephys data type

* [ENH] Added objects and rules for coordinate system sidecar file

* [ENH] Added processing microephys metadata field in rules

* [ENH] Added pharmaceuticals microephys metadata field in rules

* [ENH] Added supplementary  microephys metadata field and rules

* [ENH] Added sample microephys metadata rules

* [ENH] Added task microephys metadata rules

* Fixed example language convention

* Added microephys in schema rule modality

* Corrected SampleThickness to SliceThickness

* Add yaml document separator for microephys rules file

It is optional, and adding does not solve anything but makes it
consistent with the other files in the folder

* BF: should be a dict, not a list of dicts

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Yaroslav Halchenko <[email protected]>
Comment on lines +26 to +28
alpha_rotation: recommended
beta_rotation: recommended
gamma_rotation: recommended
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: switch out for yaw, pitch and roll

* origin/master: (288 commits)
  chore(deps): bump codecov/codecov-action from 4 to 5 (#1989)
  chore: Bump schema post-dev version
  schema-0.11.3.post3
  chore(ci): Fix pytest call for make_archive
  chore: Remove excess test data from bidsschematools installation directories (#1985)
  feat(cli): Add tool for filename validation for use in pre-receive hooks (#1986)
  rm COC (#1979)
  chore: Bump schema post-dev version
  schema-0.11.3.post2
  Update CONTRIBUTING.md (#1978)
  fix(schema): Check SliceTiming length against SliceEncodingDirection
  fix(schema): Do not warn about missing events for task-noise
  fix(schema): Check for existence of stim_files in beh.tsv
  chore: Use GITHUB_REF(_NAME) correctly
  chore: Debug environment
  chore: Bump schema post-dev version
  schema-0.11.3.post1
  chore: Output version, set correct env var
  chore: Update publish_schema to publish post-releases
  fix(schema): Include changes to schema that do not modify spec
  ...

 Conflicts:
	mkdocs.yml
	src/schema/objects/columns.yaml
	src/schema/objects/metadata.yaml
	src/schema/objects/modalities.yaml
	src/schema/objects/suffixes.yaml
	src/schema/rules/modalities.yaml


```
samples.tsv
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use MACROS___make_filetree_example

* origin/master: (26 commits)
  Update tools/schemacode/src/bidsschematools/types/namespace.py
  fail build on macro fails
  graceful error when failing to load invalid yml
  fix
  Update mkdocs.yml
  fix
  more seg
  Update src/modality-specific-files/electroencephalography.md
  fix channel, elec, sim recording
  fix key value and in-out
  fix tabular and seg
  reset
  fix internal links
  enh: Link all DICOM tags following standard pattern
  chore(links): Fix http -> https 301 redirects
  Update Release_Protocol.md (#1997)
  schema-1.0.1-dev
  schema-1.0.0
  [SCHEMA] Allow physio files for anat datatype (#1961)
  FIX: Add 1.10.0 to known versions list (#1994)
  ...
@@ -342,12 +342,12 @@ split:
display_name: Split
description: |
In the case of long data recordings that exceed a file size of 2Gb,
`.fif` files are conventionally split into multiple parts.
Each of these files has an internal pointer to the next file.
`.fif`, `.nwb`, `.nix` files are conventionally split into multiple parts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NWB files that exceed 2GB are not "conventionally" split into multiple parts. I don't know if this is true for NIX but I doubt it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +26 to +28
alpha_rotation: recommended
beta_rotation: recommended
gamma_rotation: recommended
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
alpha_rotation: recommended
beta_rotation: recommended
gamma_rotation: recommended
yaw: recommended
pitch: recommended
roll: recommended

* cutting line lenghts

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* marking json examples

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* microephysChannels

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Comment on lines +145 to +149
probe_id hemisphere x y z type material location
p023 left -11.87 -1.30 -3.37 utah-array iridium-oxide V1
p023 left -11.64 0.51 -4.20 utah-array iridium-oxide V2
p021 left -12.11 -3.12 -2.54 utah-array iridium-oxide V4
p021 left -9.94 -1.19 -2.86 utah-array iridium-oxide V3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
probe_id hemisphere x y z type material location
p023 left -11.87 -1.30 -3.37 utah-array iridium-oxide V1
p023 left -11.64 0.51 -4.20 utah-array iridium-oxide V2
p021 left -12.11 -3.12 -2.54 utah-array iridium-oxide V4
p021 left -9.94 -1.19 -2.86 utah-array iridium-oxide V3
probe_id hemisphere x y z type material location
p023 left -11.87 -1.30 -3.37 utah-array iridium-oxide V1
p023 left -11.64 0.51 -4.20 utah-array iridium-oxide V2
p021 left -12.11 -3.12 -2.54 utah-array iridium-oxide V4
p021 left -9.94 -1.19 -2.86 utah-array iridium-oxide V3

Comment on lines +565 to +588
```
dataset_description.json
tasks.tsv
tasks.json
participants.tsv
sub-A/
ses-20220101/
ephys/
sub-A_ses-20220101_task-nosepoke_ephys.nix
sub-A_ses-20220101_task-nosepoke_ephys.json
sub-A_ses-20220101_task-nosepoke_events.tsv
sub-A_ses-20220101_task-rest_ephys.nix
sub-A_ses-20220101_task-rest_ephys.json
sub-A_ses-20220101_channels.tsv
sub-A_ses-20220101_electrodes.tsv
sub-A_ses-20220101_probes.tsv
ses-20220102/
ephys/
sub-A_ses-20220102_task-rest_ephys.nix
sub-A_ses-20220102_task-rest_ephys.json
sub-A_ses-20220102_channels.tsv
sub-A_ses-20220102_electrodes.tsv
sub-A_ses-20220102_probes.tsv
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to do: use filetree macro

Comment on lines +49 to +52
name: alpha_rotation
display_name: Alpha rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name: alpha_rotation
display_name: Alpha rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
name: roll
display_name: Roll rotation
description: |
Clockwise rotation when looking down at the probe.

@@ -45,6 +45,40 @@ age:
"Description": "Subject age in postnatal years",
"Units": "year",
}
alpha_rotation:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
alpha_rotation:
roll:

Comment on lines +75 to +79
beta_rotation:
name: beta_rotation
display_name: Beta rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
Copy link
Contributor

@bendichter bendichter Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
beta_rotation:
name: beta_rotation
display_name: Beta rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
pitch:
name: pitch
display_name: Pitch rotation
description: Rotation in the direction of the electrode face, in degrees.

Comment on lines +302 to +306
gamma_rotation:
name: gamma_rotation
display_name: Gamma rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
Copy link
Contributor

@bendichter bendichter Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
gamma_rotation:
name: gamma_rotation
display_name: Gamma rotation
description: |
Euler angle to match probe extension dimensions (width, height, depth) to global x, y, z coordinates.
yaw:
name: yaw
display_name: Yaw rotation
description: Rotation clockwise when looking down.

Comment on lines +371 to +378
height__probes:
name: height
display_name: Height
description: |
Physical height of the probe, for example, '5'.
This dimension corresponds to the y’axis of the Euler transformation defined by
alpha, beta and gamma rotations values below.
type: number
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
height__probes:
name: height
display_name: Height
description: |
Physical height of the probe, for example, '5'.
This dimension corresponds to the y’axis of the Euler transformation defined by
alpha, beta and gamma rotations values below.
type: number

I don't understand what this is supposed to be

Comment on lines +1116 to +1123
width__probes:
name: width
display_name: Width
description: |
Physical width of the probe, for example, '5'.
This dimension corresponds to the x’axis of the Euler transformation defined by
alpha, beta and gamma rotations values.
type: number
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
width__probes:
name: width
display_name: Width
description: |
Physical width of the probe, for example, '5'.
This dimension corresponds to the x’axis of the Euler transformation defined by
alpha, beta and gamma rotations values.
type: number

I don't understand what this is supposed to be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.