Read epochs info from nwb2 files

[sgratiy]

Currently epochs are inferred from traces. Occasionally, inferring fails, failing the experiment. The more complex the stimulus the more issues we face with inferring the epochs. Thus, reading epoch data from the source file is highly desirable.
Epoch information actually is available in the notebooks:
dataset._data.notebook.get_value("Epochs", sweep_num, None)
but it is not part of NWB2 standard.

Describe the solution you'd like
Ephys pipeline is planning to start recording epochs into TimeIntervals table in NWB2. For such data we should certainly use existing epoch data instead of inferring it. We should still keep support for inferring epochs from traces in case NWB2 is missing epoch information.

Describe alternatives you've considered
Inferring epoch data from the notebook, but is not NWB2 standard

Additional context
The names of epochs provide in NWB file should be easily interpret able by IPFX

[timjarsky]

@sgratiy @tmchartrand

We have completed a MIES PR that generates epoch information in the TimeIntervals table. It would be helpful if a sample file was tested prior to merging the PR. I've attached a sample NWB file here that contains the TimeIntervals table.

Sst-IRES-Cre;Ai14-579951.06.03.05-compressed.zip

Here is a graphical representation of the epoch information in the TimeIntervals table

(each trace, moving down, goes a layer deeper into the epoch tree level)

[tmchartrand]

@timjarsky I took a look at the example file, and the representation makes sense for the most part. It doesn't seem to correspond to this example though, as I only see intervals with tree_level 0 or 1 (maybe the plot is supposed to be separate?).

Besides that, my only question is whether we've defined and documented the vocabulary for the epoch tags. Are there existing examples of folks using this capability of the format yet, or attempts at pinning down what info to put in the tags?

A few specifics there that maybe seem a bit counterintuitive to me:

• using the same set of tags (Inserted TP, Test Pulse) for both the whole TP sequence (level 0) and the pulse itself (level 1). also not sure whether those two tags may be redundant.
• using different tags for what are essentially identical intervals depending whether the interval is within the TP or Stimset sequence (Baseline vs amplitude 0 square pulse, and pulse vs Type=Square pulse). I'd maybe suggest using the same "property=" tags in both cases, with an added tag giving the purpose/meaning of the epoch when within the TP (e.g. Baseline or Test Pulse / pulse)
• maybe all tags should be in the form "property=x" for consistency? this would mean changing those that are currently just a name to something like "Name=Baseline"

Feel free to follow up with a direct chat if you want!

[timjarsky]

@tmchartrand thank you for your feedback. We are going to implement some of your suggested changes and share a new file here when it's available. That might be a good time to follow up with a direct chat.

To your questions:

That example is for one of our toughest stimulus types to generate epoch info for (pulse trains; used in developing and testing epoch info). The file I shared is from IVSCC and they don't utilize that stimulus.

As far as I know, we are the first to use the interval table and epoch tag for whole cell electrophys data. Here is our documentation https://alleninstitute.github.io/MIES/epoch_information.html?highlight=epoch