Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Zarr tutorial #1835

Merged
merged 10 commits into from
Jan 29, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,10 @@

Note that the Zarr native storage formats are optimized for storage in cloud storage
(e.g., S3). For very large files, Zarr will create many files which can lead to
issues for traditional file systems (that are not cloud object stores) due to
limitations on the number of files per directory (this affects local disk,
GDrive, Dropbox etc.).
issues for traditional file system (that are not cloud object stores) due to limitations
on the number of files per directory (this affects local disk, GDrive, Dropbox etc.).

Zarr read and write is provided by the :hdmf-zarr:`hdmf-zarr package<>`. First, create an
Zarr read and write is provided by the :hdmf-zarr:`hdmf-zarr` package. First, create an
bendichter marked this conversation as resolved.
Show resolved Hide resolved
an NWBFile using PyNWB.
"""

Expand All @@ -41,8 +40,8 @@
# Like HDF5, Zarr provides options to chunk and compress datasets. To leverage these
# features, replace all :py:class:`~hdmf.backends.hdf5.h5_utils.H5DataIO` with the analogous
# :py:class:`~hdmf_zarr.utils.ZarrDataIO`, which takes compressors specified by the
# `numcodecs` library. For example, to create a :py:class:`.TimeSeries`
# with a Zarr backend, use the following:
# :py:mod:`numcodecs` library. For example, here is an example :py:class:`.TimeSeries`
# where the ``data`` Dataset is compressed with a Blosc-zstd compressor:

from numcodecs import Blosc
from hdmf_zarr import ZarrDataIO
Expand All @@ -55,7 +54,7 @@
)

#######################################################################################
# Now add it to the `NWBFile`.
# Now add it to the :py:class:`.NWBFile`.

nwbfile.add_acquisition(
TimeSeries(
Expand All @@ -70,7 +69,7 @@
# Writing to Zarr
# ---------------
# To write NWB files to Zarr, replace the :py:class:`~pynwb.NWBHDF5IO` with
# :py:class:`hdmf_zarr.nwb.NWBZarrIO` for read/write
# :py:class:`hdmf_zarr.nwb.NWBZarrIO`.

from hdmf_zarr.nwb import NWBZarrIO
import os
Expand All @@ -81,9 +80,10 @@
io.write(nwbfile)

#######################################################################################
# The main reason for using the absolute_path here is for testing purposes to ensure
# links and references work as expected. Otherwise, using the relative path here instead
# is fine.
# .. note::
# The main reason for using the ``absolute_path`` here is for testing purposes to
# ensure links and references work as expected. Otherwise, using the relative path
# here instead is fine.
#
# Reading from Zarr
# -----------------
Expand All @@ -96,3 +96,4 @@
#######################################################################################
# .. note::
# For more information, see the :hdmf-zarr:`hdmf-zarr documentation<>`.

1 change: 1 addition & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ def __call__(self, filename):
'nwbwidgets': ("https://nwb-widgets.readthedocs.io/en/latest/", None),
'nwb-overview': ("https://nwb-overview.readthedocs.io/en/latest/", None),
'hdmf-zarr': ("https://hdmf-zarr.readthedocs.io/en/latest/", None),
'numcodecs': ("https://numcodecs.readthedocs.io/en/latest/", None),
}

extlinks = {
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading