Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: error when validating ElectricalSeries in NWB Zarr file #222

Open
3 tasks done
stephprince opened this issue Oct 23, 2024 · 3 comments
Open
3 tasks done

[Bug]: error when validating ElectricalSeries in NWB Zarr file #222

stephprince opened this issue Oct 23, 2024 · 3 comments
Assignees
Labels
category: bug errors in the code or code behavior
Milestone

Comments

@stephprince
Copy link
Contributor

stephprince commented Oct 23, 2024

What happened?

Ran into this error when debugging Zarr support in the NWB inspector. Read and write work fine, but validation of a Zarr file with an ElectricalSeries results in an error.

May be related to #221, but the stack trace looked slightly different so reporting here.

Steps to Reproduce

import datetime
import uuid
import numpy as np
import pynwb

from hdmf_zarr import NWBZarrIO
from pynwb import NWBFile
from pynwb.ecephys import ElectricalSeries


nwbfile = NWBFile(
    session_description="",
    identifier=str(uuid.uuid4()),
    session_start_time=datetime.datetime.now().astimezone()
)

# create an electrode group
device = nwbfile.create_device(name="device")
electrode_group = nwbfile.create_electrode_group(
    name="shank0",
    description="electrode group for shank 0",
    device=device,
    location="brain area",
)

# add electrodes to the electrode table
nwbfile.add_electrode(group=electrode_group, location="brain area",)
nwbfile.add_electrode(group=electrode_group, location="brain area",)

# create an electrical series
all_table_region = nwbfile.create_electrode_table_region(
    region=list(range(2)),
    description="all electrodes",
)

raw_data = np.random.randn(50, 2)
raw_electrical_series = ElectricalSeries(
    name="ElectricalSeries",
    description="Raw acquisition traces",
    data=raw_data,
    electrodes=all_table_region,
    starting_time=0.0,  # timestamp of the first sample in seconds relative to the session start time
    rate=20000.0,  # in Hz
)
nwbfile.add_acquisition(raw_electrical_series)

# write the NWBFile
nwbfile_path = "test_validation.nwb.zarr"
with NWBZarrIO(path=nwbfile_path, mode="w") as write_io:
    write_io.write(nwbfile)

# validate the NWBFile
with NWBZarrIO(path=nwbfile_path, mode="r") as read_io:
    invalidations = pynwb.validate(io=read_io)

Traceback

Traceback (most recent call last):
  File "/Users/smprince/Documents/code/nwbinspector/pr_debugging.py", line 53, in <module>
    invalidations = pynwb.validate(io=read_io)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/utils.py", line 672, in func_call
    return func(**pargs)
           ^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/pynwb/validate.py", line 145, in validate
    validation_errors = _validate_helper(io=io, namespace=namespace or CORE_NAMESPACE)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/pynwb/validate.py", line 29, in _validate_helper
    return validator.validate(builder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 299, in validate
    return validator.validate(builder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 479, in validate
    errors.extend(self.__validate_children(builder))
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 505, in __validate_children
    yield from self.__validate_child_builder(child_spec, child_builder, parent_builder)
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 555, in __validate_child_builder
    yield from child_validator.validate(child_builder)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 479, in validate
    errors.extend(self.__validate_children(builder))
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 505, in __validate_children
    yield from self.__validate_child_builder(child_spec, child_builder, parent_builder)
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 555, in __validate_child_builder
    yield from child_validator.validate(child_builder)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 479, in validate
    errors.extend(self.__validate_children(builder))
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 505, in __validate_children
    yield from self.__validate_child_builder(child_spec, child_builder, parent_builder)
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 555, in __validate_child_builder
    yield from child_validator.validate(child_builder)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 438, in validate
    if not check_type(self.spec.dtype, dtype, string_format):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 89, in check_type
    return received in __allowable[expected]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'list'

Operating System

macOS

Python Executable

Conda

Python Version

3.12

Package Versions

hdmf_zarr 0.9.0
pynwb 2.8.2

Code of Conduct

@oruebel
Copy link
Contributor

oruebel commented Oct 24, 2024

  File "/Users/smprince/anaconda3/envs/nwbinspector/lib/python3.12/site-packages/hdmf/validate/validator.py", line 438, in validate
    if not check_type(self.spec.dtype, dtype, string_format):
...
TypeError: unhashable type: 'list'

Would be useful to know for what field this dtype check is failing. lists usually occur as a dtype in the case of compound data types.

@stephprince
Copy link
Contributor Author

stephprince commented Oct 31, 2024

This example is now giving me a different trace, similar to the one mentioned here. I'm not sure what has changed.

@mavaylon1 mavaylon1 added the category: bug errors in the code or code behavior label Nov 7, 2024
@mavaylon1 mavaylon1 added this to the 1.0.0 milestone Nov 7, 2024
@mavaylon1
Copy link
Contributor

@stephprince Let's talk about this at the issue-a-athon

@rly rly modified the milestones: 1.0.0, 0.11.0 Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: bug errors in the code or code behavior
Projects
None yet
Development

No branches or pull requests

4 participants