Support NWB schema 2.8.0 (#2001)

Co-authored-by: Ryan Ly <[email protected]>
Co-authored-by: Steph Prince <[email protected]>

CHANGELOG.md

@@ -1,26 +1,32 @@
 # PyNWB Changelog
 
-## PyNWB 2.8.3 (November 19, 2024)
+## PyNWB 3.0.0 (Upcoming)
 
 ### Enhancements and minor changes
-* Added `NWBHDF5IO.read_nwb` convenience method to simplify reading an NWB file. @h-mayorquin [#1979](https://github.com/NeurodataWithoutBorders/pynwb/pull/1979)
-* Removed unused references to region references and builders in preparation for changes in HDMF 4.0. @rly [#1991](https://github.com/NeurodataWithoutBorders/pynwb/pull/1991)
+- Added support for NWB schema 2.8.0. @rly [#2001](https://github.com/NeurodataWithoutBorders/pynwb/pull/2001)
+  - Removed `SpatialSeries.bounds` field that was not functional. This will be fixed in a future release. @rly [#1907](https://github.com/NeurodataWithoutBorders/pynwb/pull/1907), [#1996](https://github.com/NeurodataWithoutBorders/pynwb/pull/1996)
+  - Added support for `NWBFile.was_generated_by` field. @stephprince [#1924](https://github.com/NeurodataWithoutBorders/pynwb/pull/1924)
+  - Added support for `model_number`, `model_name`, and `serial_number` fields to `Device`. @stephprince [#1997](https://github.com/NeurodataWithoutBorders/pynwb/pull/1997)
+  - Deprecated `EventWaveform` neurodata type. @rly [#1940](https://github.com/NeurodataWithoutBorders/pynwb/pull/1940)
+  - Deprecated `ImageMaskSeries` neurodata type. @rly [#1941](https://github.com/NeurodataWithoutBorders/pynwb/pull/1941)
 
-### Documentation and tutorial enhancements
-- Added documentation example for `SpikeEventSeries`. @stephprince [#1983](https://github.com/NeurodataWithoutBorders/pynwb/pull/1983)
-- Added documentation example for `AnnotationSeries`. @stephprince [#1989](https://github.com/NeurodataWithoutBorders/pynwb/pull/1989)
-- Added documentation example for `DecompositionSeries`. @stephprince [#1981](https://github.com/NeurodataWithoutBorders/pynwb/pull/1981)
+## PyNWB 2.8.3 (November 19, 2024)
 
-### Performance
+### Enhancements and minor changes
+- Added `NWBHDF5IO.read_nwb` convenience method to simplify reading an NWB file. @h-mayorquin [#1979](https://github.com/NeurodataWithoutBorders/pynwb/pull/1979)
+- Removed unused references to region references and builders in preparation for changes in HDMF 4.0. @rly [#1991](https://github.com/NeurodataWithoutBorders/pynwb/pull/1991)
+- Made gain an optional argument for PatchClampSeries to match the schema. @stephprince [#1975](https://github.com/NeurodataWithoutBorders/pynwb/pull/1975)
+- Added warning when writing files with `NWBHDF5IO` without the `.nwb` extension. @stephprince [#1978](https://github.com/NeurodataWithoutBorders/pynwb/pull/1978)
 - Cache global type map to speed import 3X. @sneakers-the-rat [#1931](https://github.com/NeurodataWithoutBorders/pynwb/pull/1931)
 
 ### Bug fixes
 - Fixed bug in how `ElectrodeGroup.__init__` validates its `position` argument. @oruebel [#1770](https://github.com/NeurodataWithoutBorders/pynwb/pull/1770)
 - Changed `SpatialSeries.reference_frame` from required to optional as specified in the schema. @rly [#1986](https://github.com/NeurodataWithoutBorders/pynwb/pull/1986)
 
-### Enhancements and minor changes
-- Made gain an optional argument for PatchClampSeries to match the schema. @stephprince [#1975](https://github.com/NeurodataWithoutBorders/pynwb/pull/1975)
-- Added warning when writing files with `NWBHDF5IO` without the `.nwb` extension. @stephprince [#1978](https://github.com/NeurodataWithoutBorders/pynwb/pull/1978)
+### Documentation and tutorial enhancements
+- Added documentation example for `SpikeEventSeries`. @stephprince [#1983](https://github.com/NeurodataWithoutBorders/pynwb/pull/1983)
+- Added documentation example for `AnnotationSeries`. @stephprince [#1989](https://github.com/NeurodataWithoutBorders/pynwb/pull/1989)
+- Added documentation example for `DecompositionSeries`. @stephprince [#1981](https://github.com/NeurodataWithoutBorders/pynwb/pull/1981)
 
 ## PyNWB 2.8.2 (September 9, 2024)
 
@@ -60,7 +66,7 @@
 ## PyNWB 2.7.0 (May 2, 2024)
 
 ### Enhancements and minor changes
-- Added `bounds` field to `SpatialSeries` to set optional boundary range (min, max) for each dimension of data. @mavaylon1 [#1869](https://github.com/NeurodataWithoutBorders/pynwb/pull/1869/files)
+- Added `bounds` field to `SpatialSeries` to set optional boundary range (min, max) for each dimension of data. @mavaylon1 [#1869](https://github.com/NeurodataWithoutBorders/pynwb/pull/1869)
 - Added support for NWB schema 2.7.0. See [2.7.0 release notes](https://nwb-schema.readthedocs.io/en/latest/format_release_notes.html) for details
 - Deprecated `ImagingRetinotopy` neurodata type. @rly [#1813](https://github.com/NeurodataWithoutBorders/pynwb/pull/1813)
 - Modified `OptogeneticSeries` to allow 2D data, primarily in extensions of `OptogeneticSeries`. @rly [#1812](https://github.com/NeurodataWithoutBorders/pynwb/pull/1812)

docs/gallery/domain/ecephys.py

@@ -79,10 +79,16 @@
 #
 # The electrodes table references a required :py:class:`~pynwb.ecephys.ElectrodeGroup`, which is used to represent a
 # group of electrodes. Before creating an :py:class:`~pynwb.ecephys.ElectrodeGroup`, you must define a
-# :py:class:`~pynwb.device.Device` object using the method :py:meth:`.NWBFile.create_device`.
-
+# :py:class:`~pynwb.device.Device` object using the method :py:meth:`.NWBFile.create_device`. The fields
+# ``description``, ``manufacturer``, ``model_number``, ``model_name``, and ``serial_number`` are optional, but
+# recommended.
 device = nwbfile.create_device(
-    name="array", description="the best array", manufacturer="Probe Company 9000"
+    name="array",
+    description="A 12-channel array with 4 shanks and 3 channels per shank",
+    manufacturer="Array Technologies",
+    model_number="PRB_1_4_0480_123",
+    model_name="Neurovoxels 0.99",
+    serial_number="1234567890",
 )
 
 #######################
@@ -383,7 +389,7 @@
 
 ######################################
 # If you do not want to store the raw voltage traces and only the waveform 'snippets' surrounding spike events,
-# you should use :py:class:`~pynwb.ecephys.SpikeEventSeries` objects.
+# you should store the snippets with :py:class:`~pynwb.ecephys.SpikeEventSeries` objects.
 #
 # NWB also provides a way to store features of spikes, such as principal components, using the
 # :py:class:`~pynwb.ecephys.FeatureExtraction` class.

docs/gallery/domain/ophys.py

@@ -86,12 +86,17 @@
 #     :align: center
 #
 # Create a :py:class:`~pynwb.device.Device` named ``"Microscope"`` in the :py:class:`~pynwb.file.NWBFile` object. Then
-# create an  :py:class:`~pynwb.ophys.OpticalChannel` named ``"OpticalChannel"``.
+# create an  :py:class:`~pynwb.ophys.OpticalChannel` named ``"OpticalChannel"``. The fields
+# ``description``, ``manufacturer``, ``model_number``, ``model_name``, and ``serial_number`` are optional, but
+# recommended.
 
 device = nwbfile.create_device(
     name="Microscope",
     description="My two-photon microscope",
-    manufacturer="The best microscope manufacturer",
+    manufacturer="Loki Labs",
+    model_number="ABC-123",
+    model_name="Loki 1.0",
+    serial_number="1234567890",
 )
 optical_channel = OpticalChannel(
     name="OpticalChannel",

docs/gallery/domain/plot_behavior.py

@@ -105,17 +105,13 @@
 #
 # For position data ``reference_frame`` indicates the zero-position, e.g.
 # the 0,0 point might be the bottom-left corner of an enclosure, as viewed from the tracking camera.
-# In :py:class:`~pynwb.behavior.SpatialSeries`, the ``bounds`` field allows the user to set
-# the boundary range, i.e.,  (min, max), for each dimension of ``data``. The units are the same as in ``data``.
-# This field does not enforce a boundary on the dataset itself.
 
 timestamps = np.linspace(0, 50) / 200
 
 position_spatial_series = SpatialSeries(
     name="SpatialSeries",
     description="Position (x, y) in an open field.",
     data=position_data,
-    bounds=[(0,50), (0,50)],
     timestamps=timestamps,
     reference_frame="(0,0) is bottom left corner",
 )

docs/gallery/general/plot_file.py

@@ -55,7 +55,6 @@
 
     * **Optical physiology and imaging:** :py:class:`~pynwb.image.ImageSeries` is the base type
       for image recordings and is further refined by the
-      :py:class:`~pynwb.image.ImageMaskSeries`,
       :py:class:`~pynwb.image.OpticalSeries`,
       :py:class:`~pynwb.ophys.OnePhotonSeries`, and
       :py:class:`~pynwb.ophys.TwoPhotonSeries` types.
@@ -92,7 +91,6 @@
       :py:class:`~pynwb.behavior.EyeTracking`.
 
     * **Extracellular electrophysiology:** :py:class:`~pynwb.ecephys.EventDetection`,
-      :py:class:`~pynwb.ecephys.EventWaveform`,
       :py:class:`~pynwb.ecephys.FeatureExtraction`,
       :py:class:`~pynwb.ecephys.FilteredEphys`,
       :py:class:`~pynwb.ecephys.LFP`.
@@ -595,8 +593,7 @@
 
 ####################
 # .. [#] Some data interface objects have a default name. This default name is the type of the data interface. For
-#    example, the default name for :py:class:`~pynwb.ophys.ImageSegmentation` is "ImageSegmentation" and the default
-#    name for :py:class:`~pynwb.ecephys.EventWaveform` is "EventWaveform".
+#    example, the default name for :py:class:`~pynwb.ophys.ImageSegmentation` is "ImageSegmentation".
 #
 # .. [#] HDF5 is the primary backend supported by NWB.
 #

src/pynwb/behavior.py

@@ -26,8 +26,6 @@ class SpatialSeries(TimeSeries):
             {'name': 'data', 'type': ('array_data', 'data', TimeSeries), 'shape': ((None, ), (None, None)), # required
              'doc': ('The data values. Can be 1D or 2D. The first dimension must be time. If 2D, there can be 1, 2, '
                      'or 3 columns, which represent x, y, and z.')},
-            {'name': 'bounds', 'type': list, 'shape': ((1, 2), (2, 2), (3, 2)), 'default': None,
-             'doc': 'The boundary range (min, max) for each dimension of data.'},
             {'name': 'reference_frame', 'type': str,
              'doc': 'description defining what the zero-position is', 'default': None},
             {'name': 'unit', 'type': str, 'doc': 'The base unit of measurement (should be SI unit)',
@@ -38,7 +36,9 @@ def __init__(self, **kwargs):
         """
         Create a SpatialSeries TimeSeries dataset
         """
-        name, data, bounds, reference_frame, unit = popargs('name', 'data', 'bounds', 'reference_frame', 'unit', kwargs)
+        name, data, reference_frame, unit = popargs(
+            'name', 'data', 'reference_frame', 'unit', kwargs
+        )
         super().__init__(name, data, unit, **kwargs)
 
         # NWB 2.5 restricts length of second dimension to be <= 3
@@ -49,7 +49,6 @@ def __init__(self, **kwargs):
                           "The second dimension should have length <= 3 to represent at most x, y, z." %
                           (name, str(data_shape)))
 
-        self.bounds = bounds
         self.reference_frame = reference_frame
 
     @staticmethod

src/pynwb/device.py

@@ -10,18 +10,41 @@ class Device(NWBContainer):
     Metadata about a data acquisition device, e.g., recording system, electrode, microscope.
     """
 
-    __nwbfields__ = ('name',
-                     'description',
-                     'manufacturer')
+    __nwbfields__ = (
+        'name',
+        'description',
+        'manufacturer',
+        'model_number',
+        'model_name',
+        'serial_number',
+    )
 
-    @docval({'name': 'name', 'type': str, 'doc': 'the name of this device'},
-            {'name': 'description', 'type': str,
-             'doc': 'Description of the device (e.g., model, firmware version, processing software version, etc.)',
-             'default': None},
-            {'name': 'manufacturer', 'type': str, 'doc': 'the name of the manufacturer of this device',
-            'default': None})
+    @docval(
+        {'name': 'name', 'type': str, 'doc': 'the name of this device'},
+        {'name': 'description', 'type': str,
+         'doc': ("Description of the device as free-form text. If there is any software/firmware associated "
+                 "with the device, the names and versions of those can be added to `NWBFile.was_generated_by`."),
+         'default': None},
+        {'name': 'manufacturer', 'type': str,
+         'doc': ("The name of the manufacturer of the device, e.g., Imec, Plexon, Thorlabs."),
+         'default': None},
+        {'name': 'model_number', 'type': str,
+         'doc': ('The model number (or part/product number) of the device, e.g., PRB_1_4_0480_1, '
+                 'PLX-VP-32-15SE(75)-(260-80)(460-10)-300-(1)CON/32m-V, BERGAMO.'),
+         'default': None},
+        {'name': 'model_name', 'type': str,
+         'doc': ('The model name of the device, e.g., Neuropixels 1.0, V-Probe, Bergamo III.'),
+         'default': None},
+        {'name': 'serial_number', 'type': str,
+         'doc': 'The serial number of the device.',
+         'default': None},
+    )
     def __init__(self, **kwargs):
-        description, manufacturer = popargs('description', 'manufacturer', kwargs)
+        description, manufacturer, model_number, model_name, serial_number = popargs(
+            'description', 'manufacturer', 'model_number', 'model_name', 'serial_number', kwargs)
         super().__init__(**kwargs)
         self.description = description
         self.manufacturer = manufacturer
+        self.model_number = model_number
+        self.model_name = model_name
+        self.serial_number = serial_number

src/pynwb/ecephys.py

@@ -120,8 +120,7 @@ class SpikeEventSeries(ElectricalSeries):
     """
     Stores "snapshots" of spike events (i.e., threshold crossings) in data. This may also be raw data,
     as reported by ephys hardware. If so, the TimeSeries::description field should describing how
-    events were detected. All SpikeEventSeries should reside in a module (under EventWaveform
-    interface) even if the spikes were reported and stored by hardware. All events span the same
+    events were detected. All events span the same
     recording channels and store snapshots of equal duration. TimeSeries::data array structure:
     [num events] [num channels] [num samples] (or [num events] [num samples] for single
     electrode).
@@ -181,6 +180,7 @@ def __init__(self, **kwargs):
 @register_class('EventWaveform', CORE_NAMESPACE)
 class EventWaveform(MultiContainerInterface):
     """
+    DEPRECATED as of NWB 2.8.0 and PyNWB 3.0.0.
     Spike data for spike events detected in raw data
     stored in this NWBFile, or events detect at acquisition
     """
@@ -193,6 +193,13 @@ class EventWaveform(MultiContainerInterface):
         'create': 'create_spike_event_series'
     }
 
+    def __init__(self, **kwargs):
+        if not self._in_construct_mode:
+            raise ValueError(
+                "The EventWaveform neurodata type is deprecated. If you are interested in using it, "
+                "please create an issue on https://github.com/NeurodataWithoutBorders/nwb-schema/issues."
+            )
+
 
 @register_class('Clustering', CORE_NAMESPACE)
 class Clustering(NWBDataInterface):

src/pynwb/file.py

@@ -261,6 +261,7 @@ class NWBFile(MultiContainerInterface, HERDManager):
                      'slices',
                      'source_script',
                      'source_script_file_name',
+                     'was_generated_by',
                      'data_collection',
                      'surgery',
                      'virus',
@@ -339,6 +340,8 @@ class NWBFile(MultiContainerInterface, HERDManager):
              'doc': 'Script file used to create this NWB file.', 'default': None},
             {'name': 'source_script_file_name', 'type': str,
              'doc': 'Name of the source_script file', 'default': None},
+            {'name': 'was_generated_by', 'type': 'array_data',
+             'doc': 'List of software package names and versions used to generate this NWB File.', 'default': None},
             {'name': 'data_collection', 'type': str,
              'doc': 'Notes about data collection and analysis.', 'default': None},
             {'name': 'surgery', 'type': str,
@@ -448,6 +451,7 @@ def __init__(self, **kwargs):
             'slices',
             'source_script',
             'source_script_file_name',
+            'was_generated_by',
             'surgery',
             'virus',
             'stimulus_notes',

src/pynwb/image.py

@@ -280,6 +280,7 @@ def __init__(self, **kwargs):
 @register_class('ImageMaskSeries', CORE_NAMESPACE)
 class ImageMaskSeries(ImageSeries):
     '''
+    DEPRECATED as of NWB 2.8.0 and PyNWB 3.0.0. 
     An alpha mask that is applied to a presented visual stimulus. The data[] array contains an array
     of mask values that are applied to the displayed image. Mask values are stored as RGBA. Mask
     can vary with time. The timestamps array indicates the starting time of a mask, and that mask
@@ -299,6 +300,11 @@ class ImageMaskSeries(ImageSeries):
                      'The device used to capture the masked ImageSeries data should be stored in the ImageSeries.'),
              'default': None},)
     def __init__(self, **kwargs):
+        if not self._in_construct_mode:
+            raise ValueError(
+                "The ImageMaskSeries neurodata type is deprecated. If you are interested in using it, "
+                "please create an issue on https://github.com/NeurodataWithoutBorders/nwb-schema/issues."
+            )
         masked_imageseries = popargs('masked_imageseries', kwargs)
         super().__init__(**kwargs)
         self.masked_imageseries = masked_imageseries

src/pynwb/io/file.py

@@ -97,6 +97,7 @@ def __init__(self, spec):
                             'session_id',
                             'slices',
                             'source_script',
+                            'was_generated_by',
                             'stimulus',
                             'surgery',
                             'virus']

tests/integration/hdf5/test_behavior.py

@@ -11,7 +11,6 @@ def setUpContainer(self):
         return SpatialSeries(
             name='test_sS',
             data=np.ones((3, 2)),
-            bounds=[(-1,1),(-1,1),(-1,1)],
             reference_frame='reference_frame',
             timestamps=[1., 2., 3.]
         )

tests/integration/hdf5/test_device.py

@@ -6,9 +6,14 @@ class TestDeviceIO(NWBH5IOMixin, TestCase):
 
     def setUpContainer(self):
         """ Return the test Device to read/write """
-        return Device(name='device_name',
-                      description='description',
-                      manufacturer='manufacturer')
+        return Device(
+            name='device_name',
+            description='description',
+            manufacturer='manufacturer',
+            model_number='model_number',
+            model_name='model_name',
+            serial_number='serial_number',
+        )
 
     def addContainer(self, nwbfile):
         """ Add the test Device to the given NWBFile """

tests/integration/hdf5/test_ecephys.py

@@ -9,7 +9,6 @@
     Clustering,
     ClusterWaveforms,
     SpikeEventSeries,
-    EventWaveform,
     EventDetection,
     FeatureExtraction,
 )
@@ -185,7 +184,7 @@ def roundtripExportContainer(self, cache_spec=False):
             return super().roundtripExportContainer(cache_spec)
 
 
-class EventWaveformConstructor(NWBH5IOFlexMixin, TestCase):
+class SpikeEventSeriesConstructor(NWBH5IOFlexMixin, TestCase):
 
     def getContainerType(self):
         return "SpikeEventSeries"
@@ -202,18 +201,16 @@ def addContainer(self):
                                     description='the first and third electrodes',
                                     table=table)
         ses = SpikeEventSeries(
-            name='test_sES',
+            name='SpikeEventSeries',
             data=((1, 1), (2, 2), (3, 3)),
             timestamps=[0., 1., 2.],
             electrodes=region
         )
 
-        ew = EventWaveform()
-        self.nwbfile.add_acquisition(ew)
-        ew.add_spike_event_series(ses)
+        self.nwbfile.add_acquisition(ses)
 
     def getContainer(self, nwbfile: NWBFile):
-        return nwbfile.acquisition['EventWaveform']
+        return nwbfile.acquisition['SpikeEventSeries']
 
 
 class ClusterWaveformsConstructor(AcquisitionH5IOMixin, TestCase):