Bug/Unexpected effect in global timestamp computation

[larsrollik]

OpenEphys GUI: 0.6.4
open-ephys-python-tools: 0.1.4

In a recently acquired session, global timestamps are miscalculated, because the events are sorted by timestamp, but the sample_number is therefore shuffled. When the global timestamp function takes the iloc[0] and iloc[-1] sample_number, then these are not the first and last sample in the acquisition.

You can see already in the head of the events dataframe that the sorting of the sample_number is random:

Out[1]: 
       line  sample_number    timestamp  ...  stream_index  stream_name state
0         1      113210450 -9906.177509  ...             1    ProbeA-AP     1
1         1      113209520 -9840.472230  ...             1    ProbeA-AP     0
2         1      113208648 -9778.864699  ...             1    ProbeA-AP     1
3         1      113206516 -9628.237112  ...             1    ProbeA-AP     0
4         1      113206244 -9609.020084  ...             1    ProbeA-AP     1
...     ...            ...          ...  ...           ...          ...   ...
25263     1       12873859   149.820794  ...             0   ProbeA-LFP     0
25264     1       12874087   149.911938  ...             0   ProbeA-LFP     1
25265     1       12510866   149.958667  ...             0   ProbeA-LFP     1
25266     1       12510889   149.967867  ...             0   ProbeA-LFP     0
25267     1       12510992   150.009067  ...             0   ProbeA-LFP     1

When checking the synchronization event counts and sample differences between the synch channels, they match perfectly, just not according to the built-in function.

Note that with the negative timestamp values, the timestamps set to -1 as not synchronized are in the middle section of the events dataframe.

[medengineer]

Hi,

This should be fixed in the latest release here:

https://pypi.org/project/open-ephys-python-tools/0.1.5/

[larsrollik]

Thank you, yesterday's release seems to fix it.

[larsrollik]

Sorry, I misread in the diff of the latest merge that the sorting of the events dataframe was changed, but this is not the case for BinaryRecording instances.

See

open-ephys-python-tools/src/open_ephys/analysis/formats/BinaryRecording.py

Lines 206 to 209 in e7a6656

	if self._version >= 0.6:
	self._events = pd.concat(df).sort_values(by=['timestamp', 'stream_index'], ignore_index=True)
	else:
	self._events = pd.concat(df).sort_values(by=['sample_number', 'stream_index'], ignore_index=True)

.

The original issue remains in https://github.com/open-ephys/open-ephys-python-tools/releases/tag/v0.1.5

[medengineer]

I see. Does changing timestamp to sample_number on line 207 above fix the issue on your end w/ version 0.1.5?

[larsrollik]

I assume so. It's done like that for the OpenEphysRecording type. But I have not written tests for it.

open-ephys-python-tools/src/open_ephys/analysis/formats/OpenEphysRecording.py

Line 290 in e7a6656

self._events = pd.concat(events).sort_values(by=['sample_number', 'stream_index'], ignore_index=True)