Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to Pydantic 2 #767

Merged
merged 6 commits into from
Mar 6, 2024
Merged

Upgrade to Pydantic 2 #767

merged 6 commits into from
Mar 6, 2024

Conversation

CodyCBakerPhD
Copy link
Member

DANDI just finished the upgrade to Pydantic 2, so we need to keep up for our live service automatic upload + GUIDE bundling

Also fixes #662 by using typing_extensions.Self

@CodyCBakerPhD CodyCBakerPhD self-assigned this Mar 2, 2024
current_location=neurodata_object.name + "/" + current_location, neurodata_object=parent
)


def _find_location_in_memory_nwbfile(neurodata_object: Container, field_name: str) -> str:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor readability change to the private methods; otherwise below there was a line that read as _find_location_in_memory_nwbfile(current_location=field_name) which looked weird until you followed the redirection and implicit logic enough to realize how the location is constructed recursively

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got you, this makes the signature more meaningful. Glad we make all of this private.

Should't the typing be NWBContainer? https://pynwb.readthedocs.io/en/stable/pynwb.core.html#pynwb.core.NWBContainer

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an NWBFile can have something that's a Container but not an NWBContainer? Maybe, maybe not - the typing and logic should apply to the most generic case however

Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems OK to me.

It is flattening the structure that we use to write the backend plus some minor changes that I agree with: field for dataset_name, creating another function for finding the location which has better signature semantics and making some properties class variables.

The most difficult thing is to read carefully through the test as the flattening of the structure implied that they have to change.

I also think that the diff in the tests resulted a bit larger than it had to be because we are testing string representations.

I will re-read the tests tomorrow just as a sanity check for my eyes and approve after as I think this is ready to go.

requirements-minimal.txt Show resolved Hide resolved
current_location=neurodata_object.name + "/" + current_location, neurodata_object=parent
)


def _find_location_in_memory_nwbfile(neurodata_object: Container, field_name: str) -> str:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got you, this makes the signature more meaningful. Glad we make all of this private.

Should't the typing be NWBContainer? https://pynwb.readthedocs.io/en/stable/pynwb.core.html#pynwb.core.NWBContainer

Copy link

codecov bot commented Mar 5, 2024

Codecov Report

Attention: Patch coverage is 93.84615% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 91.08%. Comparing base (84ed29f) to head (beabed5).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #767      +/-   ##
==========================================
- Coverage   91.17%   91.08%   -0.10%     
==========================================
  Files         121      121              
  Lines        6315     6296      -19     
==========================================
- Hits         5758     5735      -23     
- Misses        557      561       +4     
Flag Coverage Δ
unittests 91.08% <93.84%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
src/neuroconv/tools/nwb_helpers/__init__.py 100.00% <100.00%> (ø)
...nwb_helpers/_configuration_models/_base_backend.py 100.00% <100.00%> (ø)
...nwb_helpers/_configuration_models/_hdf5_backend.py 100.00% <100.00%> (ø)
..._helpers/_configuration_models/_hdf5_dataset_io.py 68.57% <100.00%> (-2.49%) ⬇️
...nwb_helpers/_configuration_models/_zarr_backend.py 100.00% <100.00%> (ø)
..._helpers/_configuration_models/_zarr_dataset_io.py 83.63% <100.00%> (-0.85%) ⬇️
.../neuroconv/tools/nwb_helpers/_configure_backend.py 95.23% <100.00%> (ø)
...roconv/tools/nwb_helpers/_dataset_configuration.py 91.66% <100.00%> (ø)
src/neuroconv/tools/testing/__init__.py 100.00% <ø> (ø)
...roconv/tools/testing/_mock/_mock_dataset_models.py 100.00% <100.00%> (ø)
... and 1 more

Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests look OK to me. This is good to go.

)


@pytest.mark.parametrize(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that this is done by pydantic now.

HDF5BackendConfiguration,
HDF5DatasetIOConfiguration,
ZarrBackendConfiguration,
ZarrDatasetIOConfiguration,
)


def mock_DatasetInfo(
def mock_HDF5DatasetIOConfiguration(
Copy link
Collaborator

@h-mayorquin h-mayorquin Mar 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mock at the beginning of the name. I think a good idea. It could also be mock_DatasetIOConfigurarion(backend: ["hdf5", "zarr"])) to save some code duplcation.

But I am not sure that is better and probably does not really matter for only two objects.

@CodyCBakerPhD CodyCBakerPhD merged commit 2a2d805 into main Mar 6, 2024
38 of 43 checks passed
@CodyCBakerPhD CodyCBakerPhD deleted the upgrade_pydantic branch March 6, 2024 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Reminder]: When Python >= 3.11, using typing.Self
2 participants