Legacy USB hardware compatibility

[samuelgarcia]

Here a naive question I wanted to ask during the today presentation.
Is it plan to port the open-ephys board usb3 to this new ONI C layer ?
Or ONI is it necessray linked to the PCIe board ?

[jonnew]

That's a great question. It's completely possible thanks largely to @aacuevas's addition of a hardware abstraction layer to the specification. This is a WIP that I need to push to this repo. Its certainly doable, but it comes down to motivation. Traditional OE hardware is compatible with both OE GUI and Bonsai. I'm not sure that we want to create an ONI driver for this hardware given this since it would largely reproduce the same functionality. I think perhaps a related question though, is if we could have USB ONI hardware that support next generation headstages. This is something I've had in mind for a while and even have an OEM base board in mind for doing it: https://numato.com/product/proteus-kintex-7-usb-3-1-development-board. This board is identical to the one we are using for PCIe, but it uses USB3 as its physical host interface. Two things would need to be done there: some USB3 IP would need to be placed on the FPGA on this board to facilitate host communication and a hardware abstraction library would need to be written to convert the FTDI API into ONI. These are the same steps we took for PCIe, but instead of FTDI, we used RIFFA (https://github.com/KastnerRG/riffa).

Let me know if this makes sense or if you were thinking about something else.

[samuelgarcia]

Thanks for the answer.

I really do think that having the same layer even for the old device would be usefull.
Of course it would largely duplicate the functionality for plugin-GUI and bonsai.

I have in mind a python layer of course (sorry).

I could so plug directly my tridesclous engine for spike sorting. :)

But more generally a python API would be super usefull for end user that would like to script custum real time processing. For instance everything related to machine learning is just easy in python. Many people use python for offline analysis having the same language for online script would be a selling point. No ? I am aware of the PythonPlugin for the OE but it is not really straightforward for a end use (like me).

Further more, a lightweight compilation in C without strong dependencies (unlike plugin-GUI) to access data packet is also another advantage. No ?

I totaly understand that you time and motivation for this. But if you have ressource to do it it could be great.

I was not aware of this roadmap about this Proteus Kintex 7 USB 3.1. It would be super great.

[aacuevas]

Another issue is that the Open Ephys USB board is not ONI-compliant. It users a different protocol with its own API (Rhythm, developed by Intan), so the driver would need to not only communicate with the hardware, but also implement a translation layer.

For Python bindings or a lightweight C build accessing the board, I would recommend taking a look at Rhythm's C API. It offers access to all featyres of the hardware. The downside is that it is somewhat low level and things like headstage discovery and calibration must be done on the client application.

[samuelgarcia]

Thanks for the answer.

If I anderstand correctly there 2 level of ONI:

1. one hardware layer compliency
2. this software C API : (also call ONI)

So the open-ephys usb3 previous generation will never have 1. but could have 2. with lot of works for translation from layer to this layer. So too much work and useless, is that right ?

Could you tell where I can find this C API for Rhythm ?

[aacuevas]

ONI is, at its core, a protocol specification, designed to offer a simple way for neuroscience applications to communicate with an heterogeneous set of hardware devices using a common set of data streams and interfaces. This repository is dedicated to the specification itself, independent of the implementations.
Our next-gen system is, thus, an ONI-compliant device, as it implements the specification.

Then there is the API, whose official name is libONI, which exposes a simple API to communicate with ONI-compliant devices. The API, on itself, communicates using the ONI-defined streams and interfaces, in a hardware-independent manner.

Those streams, however, are transported over a hardware layer. That's why we created the "onidrivers" for the API. Those are basically small libraries that provide simple instructions on what the required low-level hardware commands are to reach the ONI-defined streams and interfaces.

The structure of the whole systes is a nested set of layers:

• Real-world signals. Input (e.g.: data) and Output (e.g.: stimulation)
  • ONI-compliant devices and protocol
    • Interface hardware (USB controller, PCIe driver...) and OS kernel drivers
    • libONI's onidriver
  • libONI
• User software

Keep in mind that in our case, both our next-gen system and libONI are just our particular specifications of the standard. Other developers could develop a different API, with a completely different set of functions and dataframe management and still be able to use it with any ONI-compliant device, provided the hardware communication layers are available.

Now, back to the classic Open-Ephys device, it is not an ONI-compliant device, so the onidriver would need to implement not only the hardware communication layer, but also a way to translate from Rhythm to the ONI specification so libONI would be able to communicate with it. That means, basically, the same as developing a complete client using Rhythm from scratch. I wouldn't call it useless, and we might try that at some point but it is not a high priority endeavor at this moment.

Regarding Rhythm, you can find documentation and the source code on Intan's website for their USB2 and USB3 independent versions. We have a modified version on the classic GUI that supports both our USB2 and USB3 boards and has specific functions to those (like toggling the status LEDs). The main file of our Rhythm plugin, as well as Intan's applications source code are places where you could see how the Rhythm API is used to do device discovery and calibration, among other things.

I hope the structure of the whole project is clarified now.

[samuelgarcia]

Thank you Aaron.
This is totalyl cristal clear.
I will this code to have an over of this Rythm API.

Thanks a lot for eveything to all of you.
I will try to back on this repo in one month to make python wrapper and help for the dummy drivers.