Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework ZarrIO backend/Remove python3.7/Update HDMF and PyNWB min/Update workflows #120

Merged
merged 62 commits into from
Sep 29, 2023

Conversation

mavaylon1
Copy link
Contributor

@mavaylon1 mavaylon1 commented Sep 16, 2023

Motivation

What was the reasoning behind this change? Please explain the changes briefly.
This branch holds the re-work of the ZarrIO backend on how it handles object references. The changes are to follow the same workflow HDMF has when handling object references.

With the changes to the backend, we have a temporary ignore on dtype conversion. It is has been raised in this ticket: #120

This PR removes python 3.7, brings hdmf and pynwb to the most up to date, as well updates to the workflows.

How to test the behavior?

Show how to reproduce the new behavior (can be a bug fix or a new feature)

Run unit tests and gallery examples, specifically test_io_convert.py and plot_convert_nwb_hdf5.py

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Have you checked our Contributing document?
  • Have you ensured the PR clearly describes the problem and the solution?
  • Is your contribution compliant with our coding style? This can be checked running ruff from the source directory.
  • Have you checked to ensure that there aren't other open Pull Requests for the same change?
  • Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.

@mavaylon1 mavaylon1 changed the title Rework ZarrIO backend Rework ZarrIO backend/Remove python3.7/Update HDMF and PyNWB min/Update workflows Sep 20, 2023
Comment on lines 92 to 115
class BuilderResolverMixin(BuilderResolver):# refactor to backend/utils.py
"""
A mixin for adding to Zarr reference-resolving types
the get_object method that returns Builders
"""

def get_object(self, zarr_obj):
"""
A class that maps an Zarr object to a Builder
"""
return self.io.get_builder(zarr_obj)


class ContainerResolverMixin(ContainerResolver): # refactor to backend/utils.py
"""
A mixin for adding to Zarr reference-resolving types
the get_object method that returns Containers
"""

def get_object(self, zarr_obj):
"""
A class that maps an Zarr object to a Container
"""
return self.io.get_container(zarr_obj)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think BuilderResolverMixin and ContainerResolverMixin are not implementing any Zarr-specific functionality. I think we may be able to just import these from HDMF. As the comment says, for import we may want to move them to backend/utils.py in HDMF instead

src/hdmf_zarr/backend.py Outdated Show resolved Hide resolved
src/hdmf_zarr/backend.py Outdated Show resolved Hide resolved
src/hdmf_zarr/backend.py Outdated Show resolved Hide resolved
src/hdmf_zarr/backend.py Outdated Show resolved Hide resolved
tests/unit/test_io_convert.py Outdated Show resolved Hide resolved
@rly
Copy link
Contributor

rly commented Sep 29, 2023

This looks good to me. I'll let @oruebel do the final approval.

@rly
Copy link
Contributor

rly commented Sep 29, 2023

Thanks for your hard work on this!

CHANGELOG.md Outdated Show resolved Hide resolved
Co-authored-by: Oliver Ruebel <[email protected]>
docs/source/storage.rst Outdated Show resolved Hide resolved
src/hdmf_zarr/backend.py Outdated Show resolved Hide resolved
tests/unit/utils.py Outdated Show resolved Hide resolved
tests/unit/utils.py Outdated Show resolved Hide resolved
mavaylon1 and others added 2 commits September 29, 2023 15:57
Copy link
Contributor

@oruebel oruebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks for your patients and hard work on this PR.

@mavaylon1 mavaylon1 merged commit c262481 into dev Sep 29, 2023
21 checks passed
@mavaylon1 mavaylon1 deleted the ref branch September 29, 2023 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants