Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write scalar datasets with compound data type #1176

Merged
merged 7 commits into from
Aug 22, 2024
Merged

Conversation

stephprince
Copy link
Contributor

@stephprince stephprince commented Aug 20, 2024

Motivation

This came up in NeurodataWithoutBorders/pynwb#1770, where ElectrodeGroup.position is a scalar dataset with a compound data type. There was an issue with write, so the written dataset was not scalar and/or values would be duplicated.

Once this PR is incorporated, the related pynwb PR needs additional changes to the ElectrodeGroup position validation so that the position input is converted to a scalar with a compound data type. For example, something like:

if args_to_set['position']:
        if len(np.shape(args_to_set['position'])) > 1 or \
        (len(np.shape(args_to_set['position'])) == 1 and len(args_to_set['position']) != 3):
            raise ValueError(f"ElectrodeGroup position argument must have three elements: x, y, z,"
                            f"but received: {args_to_set['position']}")
        else:
            args_to_set['position'] = np.array(args_to_set['position'], dtype=[('x', float), ('y', float), ('z', float)])

How to test the behavior?

from pynwb import NWBFile, NWBHDF5IO
from uuid import uuid4
from datetime import datetime

nwbfile = NWBFile(
    session_description="no description.",
    session_start_time=datetime.now(),
    identifier=str(uuid4()),
)

device = nwbfile.create_device(name="Device", description="no description.")

electrode_group = nwbfile.create_electrode_group(
    name="ElectrodeGroup",
    description="no description.",
    device=device,
    location="unknown",
    position=[(1., 2., 3.)],  # --> dataset shape = (1,)
    # position=(1, 2, 3)      # --> dataset shape = (3,)
)
)

with NWBHDF5IO('test.nwb', 'w') as io:
    io.write(nwbfile)
    
with NWBHDF5IO('test.nwb', 'r') as io:
    read_nwbfile = io.read()
    
    print(read_nwbfile.electrode_groups['ElectrodeGroup'].position.shape)  # (1,)

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

@stephprince stephprince changed the title Write scalar compound datasets Write scalar datasets with compound data type Aug 20, 2024
Copy link

codecov bot commented Aug 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.89%. Comparing base (2b167ae) to head (2169fdb).
Report is 26 commits behind head on dev.

Additional details and impacted files
@@           Coverage Diff           @@
##              dev    #1176   +/-   ##
=======================================
  Coverage   88.89%   88.89%           
=======================================
  Files          45       45           
  Lines        9839     9844    +5     
  Branches     2797     2799    +2     
=======================================
+ Hits         8746     8751    +5     
  Misses        776      776           
  Partials      317      317           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@stephprince stephprince marked this pull request as ready for review August 20, 2024 20:58
@stephprince stephprince requested review from mavaylon1 and rly August 20, 2024 20:58
@stephprince stephprince added this to the 3.14.4 milestone Aug 21, 2024
src/hdmf/container.py Outdated Show resolved Hide resolved
@rly
Copy link
Contributor

rly commented Aug 22, 2024

Looks good to me!

@stephprince stephprince merged commit acc3d78 into dev Aug 22, 2024
29 checks passed
@stephprince stephprince deleted the scalar-compound-dset branch August 22, 2024 05:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants