Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README.md #614

Merged
merged 16 commits into from
Dec 10, 2024
Merged

Update README.md #614

Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
c03300b
Update README.md
ehennestad Nov 2, 2024
149c566
Update README.md
ehennestad Nov 4, 2024
d7bfd71
Merge branch 'master' into update-readme-2024
ehennestad Dec 9, 2024
d6fdeb2
Update Getting Started section of README.md
ehennestad Dec 9, 2024
bb16652
Remove "Data Dimensions" section from README.md
ehennestad Dec 9, 2024
71eb9e3
Update README.md
ehennestad Dec 9, 2024
3cc8494
Update README.md
ehennestad Dec 9, 2024
da9168b
Update README.md
ehennestad Dec 9, 2024
c77f833
Update README.md
ehennestad Dec 9, 2024
62376e1
Update README.md
ehennestad Dec 9, 2024
2de8676
Update README.md
ehennestad Dec 9, 2024
d96a1ab
Update README.md
ehennestad Dec 9, 2024
f9baa13
Merge branch 'master' into update-readme-2024
ehennestad Dec 10, 2024
c8c5f8c
Add emojis to TOC in README.md
ehennestad Dec 10, 2024
dfd54c2
Fix placement of emojis in TOC
ehennestad Dec 10, 2024
6ed877d
Small rewording - README.md
ehennestad Dec 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
221 changes: 89 additions & 132 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,196 +1,153 @@
![MatNWB Logo](logo/logo_matnwb_small.png)

[![Open in MATLAB Online](https://www.mathworks.com/images/responsive/global/open-in-matlab-online.svg)](https://matlab.mathworks.com/open/github/v1?repo=NeurodataWithoutBorders/matnwb&file=tutorials/basicUsage.mlx)
[![codecov](https://codecov.io/gh/NeurodataWithoutBorders/matnwb/branch/master/graph/badge.svg?token=apA7F24NsO)](https://codecov.io/gh/NeurodataWithoutBorders/matnwb)
[![Run tests](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_tests.yml/badge.svg)](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_tests.yml?query=event%3Apush+branch%3Amaster)
[![Codespell](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_codespell.yml/badge.svg?branch=master)](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_codespell.yml?query=event%3Apush+branch%3Amaster)

MatNWB is a Matlab interface for reading and writing Neurodata Without Borders (NWB) 2.x files.

## Setup

### Step 1: Download MatNWB

Download the current release of MatNWB from the [MatNWB releases page](https://github.com/NeurodataWithoutBorders/matnwb/releases) or from the [![View NeurodataWithoutBorders/matnwb on File Exchange](https://www.mathworks.com/matlabcentral/images/matlab-file-exchange.svg)](https://www.mathworks.com/matlabcentral/fileexchange/67741-neurodatawithoutborders-matnwb). You can also check out the latest development version via
<h1 align="center">
<br>
<a href="logo/logo_matnwb_small.png"><img src="logo/logo_matnwb.png" alt="MatNWB Logo" width="300"></a>
</h1>

<h4 align="center">MatNWB is a Matlab interface for reading and writing Neurodata Without Borders (NWB) 2.x files.</h4>

<h4 align="center">
<a href="https://github.com/NeurodataWithoutBorders/matnwb/releases/latest">
<img src="https://img.shields.io/github/v/release/NeurodataWithoutBorders/matnwb?label=version" alt="Version">
</a>
<a href="https://matlab.mathworks.com/open/github/v1?repo=NeurodataWithoutBorders/matnwb&file=tutorials/basicUsage.mlx">
<img src="https://www.mathworks.com/images/responsive/global/open-in-matlab-online.svg" alt="MATLAB Online">
</a>
<a href="https://codecov.io/gh/NeurodataWithoutBorders/matnwb">
<img src="https://codecov.io/gh/NeurodataWithoutBorders/matnwb/branch/master/graph/badge.svg?token=apA7F24NsO" alt="Codecov">
</a>
<a href="https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_tests.yml?query=event%3Apush+branch%3Amaster">
<img src="https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_tests.yml/badge.svg?branch=master" alt="Run tests">
</a>
<a href="https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_codespell.yml?query=event%3Apush+branch%3Amaster">
<img src="https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_codespell.yml/badge.svg?branch=master" alt="Codespell">
</a>
</h4>

<p align="center">
⚙️ <a href="#installation">Installation</a> •
🚀 <a href="#getting-started">Getting Started</a> •
🎓 <a href="#tutorials">Tutorials</a> •
💡 <a href="#use-cases">Use Cases</a> •
🤝 <a href="#contributing">Contributing</a> •
📄 <a href="#citing-nwb">Citing NWB</a>
</p>


## Installation
Download the current release of MatNWB from the [Releases page](https://github.com/NeurodataWithoutBorders/matnwb/releases) or from [![View NeurodataWithoutBorders/matnwb on File Exchange](https://www.mathworks.com/matlabcentral/images/matlab-file-exchange.svg)](https://www.mathworks.com/matlabcentral/fileexchange/67741-neurodatawithoutborders-matnwb). You can also check out the latest development version via

```bash
git clone https://github.com/NeurodataWithoutBorders/matnwb.git
```

### Step 2a: Reading from a NWB File

If you wish to read from a NWB file, you can do so using the `nwbRead` command:

From MATLAB's command window, add MatNWB to the [search path](https://www.mathworks.com/help/matlab/matlab_env/what-is-the-matlab-search-path.html).
```matlab
File = nwbRead('/path/to/file.nwb');
addpath('path/to/matnwb');
```

The returned NwbFile object provides an in-memory view of the underlying NWB data. For more information, see the [NWB Overview Documentation](https://nwb-overview.readthedocs.io/en/latest/file_read/file_read.html#reading-with-matnwb)

### Step 2b: Writing a NWB File


From the MATLAB command line, add MatNWB to the path. The generated classes are normally placed in the `+types` subdirectory in the MatNWB installation directory. As MATLAB [packages](https://www.mathworks.com/help/matlab/matlab_oop/scoping-classes-with-packages.html), these generated classes comprise the building blocks you will need to write your NWB file.
## Getting Started
The MatNWB 📖 documentation can be found at https://matnwb.readthedocs.io. If you find MatNWB useful, please come back later and leave us a star ⭐

### Reading from an NWB File
If you wish to read from an NWB file, you can do so using the `nwbRead` command:
```matlab
addpath('path/to/matnwb');
nwbFile = nwbRead('/path/to/file.nwb');
```

Once you have configured your NWB File, you may write the `NwbFile` object to disk using the `nwbExport` function.
The returned NwbFile object provides an in-memory view of the underlying NWB data. For more information, see the [MatNWB Documentation](https://matnwb.readthedocs.io/en/latest/pages/getting_started/file_read.html#reading-with-matnwb)

### Writing an NWB File
The building blocks of an NWB file are the neurodata types defined in the [NWB Format Specification](https://nwb-schema.readthedocs.io/en/latest/). In MatNWB, these types are autogenerated from the schema specifications and located as classes in the [`+types`](https://github.com/NeurodataWithoutBorders/matnwb/tree/master/%2Btypes) [namespace](https://www.mathworks.com/help/matlab/matlab_oop/scoping-classes-with-packages.html) of the MatNWB root directory.

To create an NWB file, you can create neurodata objects, add them to an NwbFile object and write to disk using the `nwbExport` function (detailed examples are provided in the tutorials):
```matlab
nwbExport(NwbFile, 'path/to/file.nwb');
nwbExport(nwbFile, 'path/to/file.nwb');
```

### Extensions: Generate MatNWB Classes for Extensions

The `generateExtension` command generates extension classes given a file path to the extension's namespace. This can be useful if you need to work with NWB Extension Schemas outside of Core.
### Generate MatNWB Classes for Extensions
The `generateExtension` command generates extension classes given a file path to an extension's namespace. This can be useful if you need to work with data types from [neurodata extensions](https://nwb-extensions.github.io).

```matlab
generateExtension('schema/core/nwb.namespace.yaml', '.../my_extensions1.namespace.yaml',...);
generateExtension('some/folder/my_extension1.namespace.yaml', ...);
```

### Advanced: Generating Legacy MatNWB Classes

The `generateCore` command can generate older versions of the nwb schema.
The `generateCore` command can generate classes for older versions of the nwb schema.

```matlab
generateCore('2.1.0'); % generates classes for NWB schema version 2.1.0
```

Supported schema versions are provided in the MatNWB installation directory under `nwb-schema`.
Supported schema versions are provided in the MatNWB root directory under [`nwb-schema`](https://github.com/NeurodataWithoutBorders/matnwb/tree/master/nwb-schema).

## Tutorials

[Intro to MatNWB](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/intro.html)
## Tutorials
[Intro to MatNWB](https://matnwb.readthedocs.io/en/latest/pages/tutorials/intro.html)

[Basic File Reading](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/read_demo.html) | a demo showcase for basic visualization from a DANDI dataset.
[Basic File Reading](https://matnwb.readthedocs.io/en/latest/pages/tutorials/read_demo.html) | a demo showcase for basic visualization from a DANDI dataset.

[Extracellular Electrophysiology](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/ecephys.html) |
[Extracellular Electrophysiology](https://matnwb.readthedocs.io/en/latest/pages/tutorials/ecephys.html) |
[▶️ in MATLAB Online](https://matlab.mathworks.com/open/github/v1?repo=NeurodataWithoutBorders/matnwb&file=tutorials/ecephys.mlx) |
[YouTube walkthrough](https://www.youtube.com/watch?v=W8t4_quIl1k&ab_channel=NeurodataWithoutBorders)

[Calcium Imaging](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/ophys.html) |
[Calcium Imaging](https://matnwb.readthedocs.io/en/latest/pages/tutorials/ophys.html) |
<a href="https://matlab.mathworks.com/open/github/v1?repo=NeurodataWithoutBorders/matnwb&file=tutorials/ophys.mlx" target="_blank">▶️ in MATLAB Online</a> |
[YouTube walkthrough](https://www.youtube.com/watch?v=OBidHdocnTc&ab_channel=NeurodataWithoutBorders)

[Intracellular Electrophysiology](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/icephys.html)

[Behavior](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/behavior.html)

[Optogenetics](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/ogen.html)

[Dynamic tables](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/dynamic_tables.html)

[Images](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/images.html)
[Intracellular Electrophysiology](https://matnwb.readthedocs.io/en/latest/pages/tutorials/icephys.html)

[Advanced data write](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/dataPipe.html) | [YouTube walkthrough](https://www.youtube.com/watch?v=PIE_F4iVv98&ab_channel=NeurodataWithoutBorders)
[Behavior](https://matnwb.readthedocs.io/en/latest/pages/tutorials/behavior.html)

[Using Dynamically Loaded Filters](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/dynamically_loaded_filters.html)

[Remote read](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/remote_read.html)

[Scratch Space](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/scratch.html)

## API Documentation

For more information regarding the MatNWB API or any of the NWB Core types in MatNWB, visit the [MatNWB API Documentation pages](https://neurodatawithoutborders.github.io/matnwb/doc/index.html).


## Under the Hood
[Optogenetics](https://matnwb.readthedocs.io/en/latest/pages/tutorials/ogen.html)

NWB files are HDF5 files with data stored according to the Neurodata Without Borders (NWB) [schema](https://github.com/NeurodataWithoutBorders/nwb-schema/tree/dev/core). The schema is described in a set of YAML documents which defines the various types and their attributes.
[Dynamic tables](https://matnwb.readthedocs.io/en/latest/pages/tutorials/dynamic_tables.html)

Certain functions, like `generateCore` and `nwbRead`, automatically read these specifications and converts them to a MATLAB class file. These classes generally map directly to attributes and constraints of the types defined in the schema.
[Images](https://matnwb.readthedocs.io/en/latest/pages/tutorials/images.html)

## Sources
[Advanced data write](https://matnwb.readthedocs.io/en/latest/pages/tutorials/dataPipe.html) | [YouTube walkthrough](https://www.youtube.com/watch?v=PIE_F4iVv98&ab_channel=NeurodataWithoutBorders)

MatNWB is available online at https://github.com/NeurodataWithoutBorders/matnwb
[Using Dynamically Loaded Filters](https://matnwb.readthedocs.io/en/latest/pages/tutorials/dynamically_loaded_filters.html)

## Data Dimensions
[Remote read](https://matnwb.readthedocs.io/en/latest/pages/tutorials/remote_read.html)

NWB files use the HDF5 format to store data. There are two main differences between the way MATLAB and HDF5 represents dimensions. The first is that HDF5 is C-ordered, which means it stores data is a rows-first pattern, and the MATLAB is F-ordered, storing data in the reverse pattern, with the last dimension of the array stored consecutively. The result is that the data in HDF5 is effectively the transpose of the array in MATLAB. The second difference is that HDF5 can store 1-D arrays, but in MATLAB the lowest dimensionality of an array is 2-D. Due to differences in how MATLAB and HDF5 represent data, the dimensions of datasets are flipped when writing to/from file in MatNWB. This behavior differs depending on whether ```VectorData``` use ```DataPipe``` objects to contain the data. It's important to keep in mind the mappings below to make sure is written to and read from file as expected.
[Scratch Space](https://matnwb.readthedocs.io/en/latest/pages/tutorials/scratch.html)

[without DataPipes](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/dimensionMapNoDataPipes.html)

**Writing to File**

| Shape <br /> in MatNWB| Shape<br />in HDF5|
| :----------: | :----------: |
| (M, 1) | (M,) |
| (1, M) | (M,) |
| (P, O, N, M) | (M, N, O, P) |

**Reading from File**

| Shape <br /> in HDF5| Shape<br />in MatNWB|
| :----------: | :----------: |
| (M,) | (M,1) |
| (M, N, O, P) | (P, O, N, M) |

**NOTE:** MATLAB does not support 1D datasets. HDF5 datasets of size (M,) are loaded into MATLAB as datasets of size (M,1). To avoid changes in dimensions when writing to/from file use column vectors for 1D datasets.

[with DataPipes](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/dimensionMapWithDataPipes.html)
## Use Cases
[Basic Data Retrieval](https://matnwb.readthedocs.io/en/latest/pages/tutorials/basicUsage.html)
| showcases how one would read and process converted NWB file data to display a raster diagram.

**Writing to File**
[Conversion of Real Electrophysiology/Optophysiology Data](https://matnwb.readthedocs.io/en/latest/pages/tutorials/convertTrials.html)
| converts Electrophysiology/Optophysiology Data recorded from:
>Li, Daie, Svoboda, Druckman (2016); Data and simulations related to: Robust neuronal dynamics in premotor cortex during motor planning. Li, Daie, Svoboda, Druckman, Nature. CRCNS.org
http://dx.doi.org/10.6080/K0RB72JW

| Shape <br /> in MatNWB| Shape <br /> in HDF5|
| :----------: | :----------: |
| (M, 1) | (1, M) |
| (1, M) |(M, 1)/(M,)** |
| (P, O, N, M) | (M, N, O, P) |
Analysis examples will be added in the [dandi-example-live-scripts repo](https://github.com/NeurodataWithoutBorders/dandi-example-live-scripts)

** Use scalar as input to 'maxSize' argument to write dataset of shape (N,)

**Reading from File**
## Under the Hood
NWB files are HDF5 files with data stored according to the Neurodata Without Borders (NWB) [schema](https://github.com/NeurodataWithoutBorders/nwb-schema/tree/dev/core). The schema is described in a set of YAML documents which defines the various types and their attributes.

| Shape <br /> in HDF5| Shape<br />in MatNWB|
| :----------: | :----------: |
| (M, 1) | (1, M) |
| (1, M) | (M, 1) |
| (M,) | (M, 1) |
| (M, N, O, P) | (P, O, N, M) |
Certain functions, like `generateCore` and `nwbRead`, automatically read these specifications and converts them to a set of MATLAB class files. These classes generally map directly to attributes and constraints of the types defined in the schema.


## Caveats
The NWB schema has regular updates and is open to addition of new types along with modification of previously defined types. As such, certain type presumptions made by MatNWB may be invalidated in the future from newer NWB schema versions. Furthermore, new types may require implementations that will be missing in MatNWB until patched in.

The NWB schema has regular updates and is open to addition of new types along with modification of previously defined types. As such, certain type presumptions made by MatNWB may be invalidated in the future from a NWB schema. Furthermore, new types may require implementations that will be missing in MatNWB until patched in.

For those planning on using matnwb alongside pynwb, please keep the following in mind:
- MatNWB is dependent on the schema, which may not necessary correspond with your PyNWB schema version. Please consider overwriting the contents within MatNWB's **~/schema/core** directory with the generating PyNWB's **src/pynwb/data directory** and running generateCore to ensure compatibility between systems.
For those planning on using MatNWB alongside PyNWB, please keep the following in mind:
- MatNWB is dependent on the schema, which may not necessarily correspond with your PyNWB schema version. Please consider overwriting the contents within MatNWB's **~/schema/core** directory with the generating PyNWB's **src/pynwb/data directory** and running `generateCore` to ensure compatibility between systems.

The `master` branch in this repository is considered perpetually unstable. If you desire Matnwb's full functionality (full round-trip with nwb data), please consider downloading the more stable releases in the Releases tab. Most releases will coincide with nwb-schema releases and guarantee compatibility of new features introduced with the schema release along with backwards compatibility with all previous nwb-schema releases.
The `master` branch of this repository is considered perpetually unstable. If you want to stay on the safer side, please consider downloading the more stable releases from the [Releases](https://github.com/NeurodataWithoutBorders/matnwb/releases) page. Most releases will coincide with nwb-schema releases and guarantee compatibility of new features introduced with the schema release along with backwards compatibility with all previous nwb-schema releases.

This package reads and writes NWB 2.0 files and does not support older formats.

## Examples

[Basic Data Retrieval](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/basicUsage.html)
| showcases how one would read and process converted NWB file data to display a raster diagram.

[Conversion of Real Electrophysiology/Optophysiology Data](https://neurodatawithoutborders.github.io/matnwb/tutorials/html/convertTrials.html)
| converts Electrophysiology/Optophysiology Data recorded from:
>Li, Daie, Svoboda, Druckman (2016); Data and simulations related to: Robust neuronal dynamics in premotor cortex during motor planning. Li, Daie, Svoboda, Druckman, Nature. CRCNS.org
http://dx.doi.org/10.6080/K0RB72JW

Analysis examples will be added in the [dandi-example-live-scripts repo](https://github.com/NeurodataWithoutBorders/dandi-example-live-scripts)

## Third-party Support
The `+contrib` folder contains tools for converting from other common data formats/specifications to NWB. Currently supported data types are TDT, MWorks, and Blackrock. We are interested in expanding this section to other data specifications and would greatly value your contribution!

## Testing

Run the test suite with `nwbtest`.

## FAQ

1. "A class definition must be in an "@" directory."

Make sure that there are no "@" signs **anywhere** in your *full* file path. This includes even directories that are not part of the matnwb root path and any "@" signs that are not at the beginning of the directory path.

Alternatively, this issue disappears after MATLAB version 2017b. Installing this version may also resolve these issues. Note that the updates provided with 2017b should also be installed.


2. I Have Issues Reading From a NWB File!
## Contributing
For details on how to contribute to MatNWB, please stay tuned.

Some simple methods to troubleshoot failed NWB file reads can be found in the [NWB Overview Documentation](https://nwb-overview.readthedocs.io/en/latest/file_read/matnwb/troubleshooting.html).
## Citing NWB
* **Manuscript:** Oliver Rübel, Andrew Tritt, Ryan Ly, Benjamin K Dichter, Satrajit Ghosh, Lawrence Niu, Pamela Baker, Ivan Soltesz, Lydia Ng, Karel Svoboda, Loren Frank, Kristofer E Bouchard. (2022). The Neurodata Without Borders ecosystem for neurophysiological data science. eLife, 11:e78362. doi: https://doi.org/10.7554/eLife.78362
* **RRID:** (MatNWB, RRID:SCR_021156)
Loading