Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow selection of converters with SpikeGLXConverter Support #467

Merged
merged 9 commits into from
Oct 22, 2023

Conversation

garrettmflynn
Copy link
Member

@garrettmflynn garrettmflynn commented Oct 18, 2023

fix #174

This PR adds sub-converters to the options available to the user.

There's currently a blocking error when attempting to instantiate the custom converter on the GUIDE backend:

index.js:33 [main-process]: [2023-10-18 11:40:09,245] ERROR in app: Exception on /neuroconv/metadata [POST]
Traceback (most recent call last):
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/flask_restx/api.py", line 404, in wrapper
    resp = resource(*args, **kwargs)
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/flask/views.py", line 109, in view
    return current_app.ensure_sync(self.dispatch_request)(**kwargs)
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/flask_restx/resource.py", line 46, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/Users/garrettflynn/Documents/Github/nwb-guide/pyflask/apis/neuroconv.py", line 84, in post
    return get_metadata_schema(neuroconv_api.payload.get("source_data"), neuroconv_api.payload.get("interfaces"))
  File "/Users/garrettflynn/Documents/Github/nwb-guide/pyflask/manageNeuroconv/manage_neuroconv.py", line 240, in get_metadata_schema
    converter = instantiate_custom_converter(source_data, interfaces)
  File "/Users/garrettflynn/Documents/Github/nwb-guide/pyflask/manageNeuroconv/manage_neuroconv.py", line 203, in instantiate_custom_converter
    return CustomNWBConverter(source_data)
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/neuroconv/nwbconverter.py", line 65, in __init__
    self.data_interface_objects = {
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/neuroconv/nwbconverter.py", line 66, in <dictcomp>
    name: data_interface(**source_data[name])
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/neuroconv/datainterfaces/ecephys/spikeglx/spikeglxconverter.py", line 55, in __init__
    streams = streams or self.get_streams(folder_path=folder_path)
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/neuroconv/datainterfaces/ecephys/spikeglx/spikeglxconverter.py", line 26, in get_streams
    return SpikeGLXRecordingExtractor.get_streams(folder_path=folder_path)[0]
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py", line 71, in get_streams
    neo_reader = cls.get_neo_io_reader(cls.NeoRawIOClass, **neo_kwargs)
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py", line 64, in get_neo_io_reader
    neo_reader.parse_header()
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/neo/rawio/baserawio.py", line 179, in parse_header
    self._parse_header()
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/neo/rawio/spikeglxrawio.py", line 98, in _parse_header
    data = np.memmap(info['bin_file'], dtype='int16', mode='r', offset=0, order='C')
  File "/opt/anaconda3/envs/nwb-guide/lib/python3.9/site-packages/numpy/core/memmap.py", line 268, in __new__
    mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start)
ValueError: cannot mmap an empty file

@garrettmflynn garrettmflynn self-assigned this Oct 18, 2023
@garrettmflynn garrettmflynn changed the title Add converters Allow selection of converters w/ SpikeGLXConverter Support Oct 18, 2023
@garrettmflynn garrettmflynn changed the title Allow selection of converters w/ SpikeGLXConverter Support Allow selection of converters with SpikeGLXConverter Support Oct 18, 2023
@garrettmflynn garrettmflynn added the interface Add support for a specific NeuroConv interface / converter label Oct 18, 2023
@garrettmflynn garrettmflynn marked this pull request as ready for review October 18, 2023 19:47
@garrettmflynn
Copy link
Member Author

Fixed this issue. Was just because my testing dataset—specifically the NIDQ file for the SpikeGLX test—wasn't properly resolved

@CodyCBakerPhD
Copy link
Collaborator

CodyCBakerPhD commented Oct 22, 2023

Overall looks great as a first introduction

I like the organization within the search tab too

Two minor comments, both of which can be left as future To-Do items (raise as separate issues if you desire)

i)

I like the little hover info in general, but note that a lot of these descriptions were tailored to sphinx. Any idea on how to better parse the inter-sphinx links like seen below? Just looks kind of awkward - and a lot of those are programmatic

image

ii)

The section headers could be a little more identifiable - larger maybe? Gray background? Different format? The fact that it's not clickable is a decent indicator though

image

@CodyCBakerPhD CodyCBakerPhD merged commit 13c8e8f into main Oct 22, 2023
7 checks passed
@CodyCBakerPhD CodyCBakerPhD deleted the add-converters branch October 22, 2023 18:28
@garrettmflynn
Copy link
Member Author

For (i), what would the ideal returned string be for an entry with these inter-sphinx links? Maybe for :py:class ~spikeinterface. it would be transformed to "spikeinterface class"?

I'll get a quick fix ready for (ii) otherwise.

@CodyCBakerPhD
Copy link
Collaborator

For (i) the ~ before the mock import strips the parent package and it just renders as code-format (and bold I think?) the name of the item being referenced

E.g., :py:class ~spikeinterface.extractors.SpikeGLXRecordingExtractor would be rendered in docs as SpikeGLXRecordingExtractor but it would be a hyperlink to the API documentation on the SpikeInterface website

We don't need to go to that level of course; I would just use the name stripped after the . character

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interface Add support for a specific NeuroConv interface / converter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Ecephys Support] Add SpikeGLXConverter support
2 participants