Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data with shape (0,0,0) triggering assertion error after get chunk refactor #76

Closed
2 of 3 tasks
rly opened this issue May 28, 2024 · 0 comments · Fixed by #77
Closed
2 of 3 tasks

Data with shape (0,0,0) triggering assertion error after get chunk refactor #76

rly opened this issue May 28, 2024 · 0 comments · Fixed by #77
Assignees
Labels
category: bug errors in the code or code behavior

Comments

@rly
Copy link
Contributor

rly commented May 28, 2024

I think #68 introduced a bug where now, a dataset with shape (0,0,0) as in dandiset 000409 at path acquisition/OriginalVideoBodyCamera/data raises an assertion error when generating the RFS that the chunk reference is invalid: ('https://api.dandiarchive.org/api/assets/4bab2e40-6253-4204-89ba-89b4167d46b4/download/', None, 0)

The second element should not be None. This case was previously addressed. I'll look into this later this week or if I have time today.

Error location:

File ~/Documents/NWB/lindi/lindi/LindiH5ZarrStore/LindiH5ZarrStore.py:645, in LindiH5ZarrStore.to_reference_file_system.<locals>._process_dataset(key, item)
    640 external_array_link = zattrs_dict.get(
    641     "_EXTERNAL_ARRAY_LINK", None
    642 )
    643 if external_array_link is None:
    644     # Only add chunk references for datasets without an external array link
--> 645     self._add_chunk_info_to_refs(key, _add_ref, _add_ref_chunk)

File ~/Documents/NWB/lindi/lindi/LindiH5ZarrStore/LindiH5ZarrStore.py:485, in LindiH5ZarrStore._add_chunk_info_to_refs(self, key_parent, add_ref, add_ref_chunk)
    483 byte_offset, byte_count = _get_byte_range_for_contiguous_dataset(h5_item)
    484 key_name = ".".join("0" for _ in range(h5_item.ndim))
--> 485 add_ref_chunk(f"{key_parent}/{key_name}", (self._url, byte_offset, byte_count))

File ~/Documents/NWB/lindi/lindi/LindiH5ZarrStore/LindiH5ZarrStore.py:592, in LindiH5ZarrStore.to_reference_file_system.<locals>._add_ref_chunk(key, data)
    591 def _add_ref_chunk(key: str, data: Tuple[str, int, int]):
--> 592     assert data[1] is not None

TODO:

  • track down and fix bug
  • add a test
  • release bugfix
@rly rly self-assigned this May 28, 2024
@rly rly added the category: bug errors in the code or code behavior label May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: bug errors in the code or code behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant