Unable to load old recordings (OpenEphys < 0.5.0)

[paulbrodersen]

I have inherited a data set recorded with OpenEphys version 0.4.2.1.
I am trying to import the data using open-ephys-python-tools version 0.1.12, following the steps outlined in issue #23.
However, the session object has no recordnodes attribute:

from open_ephys.analysis import Session

session = Session("140324/154.3a_2024-03-14_11-40-15_4_nolight/")

print(session)
# ---------------------------------------------------------------------------
# AttributeError                            Traceback (most recent call last)
# Cell In[4], line 1
# ----> 1 print(session)
#
# File ~/opt/miniconda3/envs/openephys/lib/python3.9/site-packages/open_ephys/analysis/session.py:92, in Session.__str__(self)
# 85 def __str__(self):
# 86     """Returns a string with information about the Session"""
# 88     return ''.join(["\nOpen Ephys Recording Session Object\n",
#                        89                     "Directory: " + self.directory + "\n\n"
#                        90                     "<object>.recordnodes:\n"] +
#                        91                     ["  Index " + str(i) + ": " + r.__str__() + "\n"
#                                                ---> 92                       for i, r in enumerate(self.recordnodes)])
#
# AttributeError: 'Session' object has no attribute 'recordnodes'

dir(session)
# ['__class__',
#  '__delattr__',
#  '__dict__',
#  '__dir__',
#  '__doc__',
#  '__eq__',
#  '__format__',
#  '__ge__',
#  '__getattribute__',
#  '__gt__',
#  '__hash__',
#  '__init__',
#  '__init_subclass__',
#  '__le__',
#  '__lt__',
#  '__module__',
#  '__ne__',
#  '__new__',
#  '__reduce__',
#  '__reduce_ex__',
#  '__repr__',
#  '__setattr__',
#  '__sizeof__',
#  '__str__',
#  '__subclasshook__',
#  '__weakref__',
#  '_detect_record_nodes',
#  'directory',
#  'mmap_timestamps',
#  'recordings']

Attribute recordings is an empty list; attribute mmap_timestamps is a boolean.

The directory structure of the session is the following:

ls -1 140324/154.3a_2024-03-14_11-40-15_4_nolight/
# 100_ADC1.continuous
# 100_ADC2.continuous
# 100_ADC3.continuous
# 100_ADC4.continuous
# 100_CH10.continuous
# 100_CH11.continuous
# 100_CH12.continuous
# 100_CH13.continuous
# 100_CH14.continuous
# 100_CH15.continuous
# 100_CH16.continuous
# 100_CH17.continuous
# 100_CH18.continuous
# 100_CH19.continuous
# 100_CH1.continuous
# 100_CH20.continuous
# 100_CH21.continuous
# 100_CH22.continuous
# 100_CH23.continuous
# 100_CH24.continuous
# 100_CH25.continuous
# 100_CH26.continuous
# 100_CH27.continuous
# 100_CH28.continuous
# 100_CH29.continuous
# 100_CH2.continuous
# 100_CH30.continuous
# 100_CH31.continuous
# 100_CH32.continuous
# 100_CH3.continuous
# 100_CH4.continuous
# 100_CH5.continuous
# 100_CH6.continuous
# 100_CH7.continuous
# 100_CH8.continuous
# 100_CH9.continuous
# all_channels.events
# Continuous_Data.openephys
# messages.events
# SE0.spikes
# SE10.spikes
# SE11.spikes
# SE12.spikes
# SE13.spikes
# SE14.spikes
# SE15.spikes
# SE1.spikes
# SE2.spikes
# SE3.spikes
# SE4.spikes
# SE5.spikes
# SE6.spikes
# SE7.spikes
# SE8.spikes
# SE9.spikes
# settings.xml

Do you have any suggestions how to troubleshoot / work around the issue?
Thank you for your time!

[paulbrodersen]

Potentially related to issue #38 ?

[medengineer]

Hi Paul,

The loader expects the data to be within a Record Node folder. Can you try adding a "Record Node 101" intermediate directory between the root folder and the data fiels as a potential quick fix?

If that doesn't work, can you send a link to the data to [email protected] and I can structure it correctly?

Thanks!

[paulbrodersen]

Hi, thanks for the quick response.

Can you try adding a "Record Node 101" intermediate directory between the root folder and the data fiels as a potential quick fix?

Unfortunately, this does not fix the issue. While Session.recordnodes is no longer an empty list, Session.recordnodes[0].recordings is.

[paulbrodersen]

However, I did manage to load the data using spikeinterface's OpenEphysLegacyRecordingExtractor.

  import spikeinterface.extractors as se
  folder = "/path/to/data/directory/"
  rec = se.OpenEphysLegacyRecordingExtractor(folder, stream_id="CH")

Would you expect that the "legacy" recordings could still be loaded by the tools in current versions of OpenEphys?

[medengineer]

Glad to hear SpikeInterface worked.

The current version of the OpenEphys GUI has significant API/architecture changes that supporting recordings from < 0.5.0 is not supported.

Similarly, the python-tools also only support data recorded from version 0.5.0+.

The archived analysis-tools are the best option for legacy recordings for now. We will consider integrating these if possible before the 1.0 release of the python-tools.

[paulbrodersen]

The current version of the OpenEphys GUI has significant API/architecture changes that supporting recordings from < 0.5.0 is not supported.
Similarly, the python-tools also only support data recorded from version 0.5.0+.
The archived analysis-tools are the best option for legacy recordings for now. We will consider integrating these if possible before the 1.0 release of the python-tools.

If I were you, I would not support legacy formats in my current code base. Ideally, I would provide a tool and/or instructions on how to convert a legacy data set into the currently supported format(s). If that is too involved, I would document how to load data using the legacy analysis-tools in the README and/or the online docs.