Skip to content

Create NWB files by converting and combining neural data in proprietary formats and adding essential metadata.

License

Notifications You must be signed in to change notification settings

catalystneuro/neuroconv

Folders and files

NameName
Last commit message
Last commit date

Latest commit

79dd197 · Feb 11, 2023
Feb 11, 2023
Jan 26, 2023
Feb 10, 2023
Feb 10, 2023
Dec 16, 2022
Feb 7, 2023
Sep 23, 2022
Feb 10, 2023
Jul 19, 2022
Jul 26, 2019
Jul 25, 2022
Oct 3, 2022
Mar 21, 2022
Sep 19, 2022
Jul 21, 2022
Aug 19, 2022
Dec 14, 2022
Dec 7, 2022
Jan 27, 2023
Feb 7, 2023

Repository files navigation

PyPI version Full Tests Auto-release codecov documentation License

NeuroConv

NeuroConv is a package for creating NWB files by converting and combining neural data in proprietary formats and adding essential metadata.

Features:

  • Command line interface
  • Python API
  • Leverages SpikeExtractor to support conversion from a number or proprietary formats.

Installation

To install the latest stable release of neuroconv though PyPI, type:

pip install neuroconv

For more flexibility we recommend installing the latest version directly from GitHub. The following commands create an environment with all the required dependencies and the latest updates:

git clone https://github.com/catalystneuro/neuroconv
cd neuroconv
conda env create -f make_environment.yml
conda activate neuroconv_environment

Note that this will install the package in editable mode.

Finally, if you prefer to avoid conda altogether, the following commands provide a clean installation within the current environment:

pip install git+https://github.com/catalystneuro/neuroconv.git@master

Dependencies

NeuroConv relies heavily on SpikeInterface and SpikeExtractors for electrophysiology and on ROIExtractors for optophysiology data.

Catalogue

v0.9.3

This project is an ongoing effort for the Ripple U19 conversion of extracellular electrophysiology data to NWB format, including final publishing of each dataset on DANDI. Currently spans 7 major publications and over 14 TB of data on the DANDI Archive. Most of the data consists of raw recordings, LFP, spike sorted units, and behavior with can consist of a mix of mental state tracking, position tracking through mazes, and trial stimulus events.

The Shenoy lab is one of the pioneers in developing BCIs for people with paralysis. They are part of the BrainGate team and were the winners of the 2019 BCI award. They use extracellular recordings from Utah arrays and Neuropixels in primates.

v0.9.2

The Brody lab has a long history with extracellular electrophysiology experiements spanning multiple acquisition systems. This project served two purposes - to allow the conversion of older data from Neuralynx and SpikeGadgets to NWB, and also their newer, larger data using Neuropixels (SpikeGLX). These recordings, some of which exceeded more than 250 GB (several hours worth!), were paired with rich trials tables containing catagorical events and temporal stimuli.

v0.8.10

The Feldman lab utilizes a Neuropixels (SpikeGLX) system along with multiple sophisticated behavior systems for manipulating whisker stimulation in mice. These give rise to very complex trials tables tracking multiple event times throughout the experiments, including multiple event trains within trials.

v0.8.1

Hussaini Lab: hussaini-lab-to-nwb

v0.7.2

v0.7.0

Neuropixel (SpikeGLX) recordings of subjects navigating a virtual reality! Behavior contains a huge variety of NWB data types including positional and view angle over time, collision detection, and more! Paired with a specific extension for parsing experiment metadata.

Utilizing the CED recording interface, this project paired ecephys channels with optogenetic stimulation via laser pulses, and mechnical pressure stimulation over time - all of which are channels of data extracted from the common .smrx files!

Other labs that use NWB standard

For Developers

Running GIN tests locally

neuroconv verifies the integrity of all code changes by running a full test suite on short examples of real data from the formats we support. There are two classes of tests in this regard; tests/test_internals does not require any data to be present and represents the 'minimal' expected behavior for our package, whereas tests/test_on_data requires the user to both perform a full install of dependencies (pip install -r requirements-full.txt) as well as download the associated data for each modality.

Install testing dependencies

In a clean environment run:

git clone https://github.com/catalystneuro/neuroconv
cd neuroconv
pip install .[test,full]

Downloading the data

Datalad (conda install datalad) is the recommended way for downloading the data. To do this; simply call:

For electrophysiology data:

datalad install -rg https://gin.g-node.org/NeuralEnsemble/ephy_testing_data

For optical physiology data:

datalad install -rg https://gin.g-node.org/CatalystNeuro/ophys_testing_data

For behavioral data:

datalad install -rg https://gin.g-node.org/CatalystNeuro/behavior_testing_data

Test configuration file

Once the data is downloaded to your system, you must manually modify the config file (example) located in ./tests/test_on_data/gin_test_config.json so its corresponding LOCAL_PATH key points to the correct folder on your system that contains the dataset folder (e.g., ephy_testing_data for testing ecephys). The code will automatically detect that the tests are being run locally, so all you need to do ensure the path is correct to your specific system.

The output of these tests is, by default, stored in a temporary directory that is then cleaned after the tests finish running. To examine these files for quality assessment purposes, set the flag SAVE_OUTPUTS=true in the gin_test_config.json file and modify the variable OUTPUT_PATH in the respective test if necessary.

Build the documentation

For building the documentation locally, the following procedure can be followed. Create a clean environment and type the following commands in your terminal:

git clone https://github.com/catalystneuro/neuroconv
cd neuroconv
pip install -e .[docs]

These commands install both the latest version of the repo and the dependencies necessary to build the documentation. Note that the argument -e makes you install editable

Now, to build the documention issue the following command in your terminal:

sphinx-build -b html docs ./docs/_build/

This builds the html under /docs/_build/ (from your root directory, where you have installed neuroconv). This allows you to review the outcome of the process localy before commiting code.

About

Create NWB files by converting and combining neural data in proprietary formats and adding essential metadata.

Resources

License

Citation

Stars

Watchers

Forks

Languages