-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[24.1] Fix h5ad metadata #18635
[24.1] Fix h5ad metadata #18635
Conversation
Wondering if we should add a test for this, analogous to https://github.com/galaxyproject/galaxy/blob/dev/test/unit/data/datatypes/test_qiime2.py |
@bernt-matthias Yes, for sure. I add a visualization test in #18552 that inadvertently exercises this as well, and I wonder if this explains that test failure that I haven't got around to fixing up yet :) Edit: Followed up on that test failure -- that was due to the test suite not loading the h5 datatype at all, so unrelated, though I'm still +1 on a datatype test here. |
Dear all, |
How large is it? If its too large put it into https://github.com/galaxyproject/galaxy-test-data |
It is 74.0 kB |
Then have a look where |
Hi @bgruening @davelopez @bernt-matthias @dannon, |
lib/galaxy/datatypes/binary.py
Outdated
@@ -1574,7 +1574,7 @@ def _layercountsize(tmp, lennames=0): | |||
dataset.metadata.shape = (int(dataset.metadata.obs_size), int(dataset.metadata.var_size)) | |||
|
|||
def set_peek(self, dataset: DatasetProtocol, **kwd) -> None: | |||
if not dataset.dataset.purged: | |||
if not dataset.file_name_ is None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if not dataset.file_name_ is None: | |
if dataset.file_name_ is not None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did you make that change, this looks quite wrong to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it gives this error:
def set_peek(self, dataset: DatasetProtocol, **kwd) -> None:
> if dataset.dataset.purged:
E AttributeError: 'NoneType' object has no attribute 'purged'
the MockDataset class does not have purge attribute:
class MockDataset:
def __init__(self, id):
self.id = id
self.metadata = MockMetadata()
self.dataset = None
self.file_name_: Optional[str] = None
def get_file_name(self, sync_cache=True):
return self.file_name_
def set_file_name(self, file_name):
self.file_name_ = file_name
def has_data(self):
return True
def get_size(self):
return self.dataset and os.path.getsize(self.dataset.get_file_name())
```
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't adjust code for unit tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please give me a hint of what I should do?
That class does not have "purged" attribute. am I right?
lib/galaxy/datatypes/binary.py
Outdated
@@ -1596,7 +1596,7 @@ def _makelayerstrings(layer, count, names): | |||
peekstr += _makelayerstrings("uns", tmp.uns_count, tmp.uns_layers) | |||
|
|||
dataset.peek = peekstr | |||
dataset.blurb = f"Anndata file ({nice_size(dataset.get_size())})" | |||
dataset.blurb = "Anndata file" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What was wrong with the dataset size?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It gives this error:
lib/galaxy/datatypes/binary.py:1599: in set_peek
dataset.blurb = f"Anndata file ({nice_size(dataset.get_size())})"
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
size = None
def nice_size(size: Union[float, int, str, Decimal]) -> str:
"""
Returns a readably formatted string with the size
>>> nice_size(100)
'100 bytes'
>>> nice_size(10000)
'9.8 KB'
>>> nice_size(1000000)
'976.6 KB'
>>> nice_size(100000000)
'95.4 MB'
"""
try:
> size = float(size)
E TypeError: float() argument must be a string or a number, not 'NoneType'
Maybe sth is wrong with nice_size
?
it is in init.py
lib/galaxy/datatypes/binary.py
Outdated
@@ -1574,7 +1574,7 @@ def _layercountsize(tmp, lennames=0): | |||
dataset.metadata.shape = (int(dataset.metadata.obs_size), int(dataset.metadata.var_size)) | |||
|
|||
def set_peek(self, dataset: DatasetProtocol, **kwd) -> None: | |||
if not dataset.dataset.purged: | |||
if dataset.file_name_ is not None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if dataset.file_name_ is not None: | |
if not dataset.dataset.purged: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file is too large, can you create a smaller one please ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it fine now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, thanks, we'll use that later for a different type of test.
You've now tailored the code to the mock implementation and it will not actually work. If the core team decides we want to validate metadata attributes we can do this more systematically. I would be in favor of merging only the fix. |
This fix worked on my local galaxy with only changing But for making a test, the other classes (like |
You can add the required attributes to the mock class if you like, but the fix seems fine as is. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot @nilchia!
Thank you @mvdbeek! |
This PR was merged without a "kind/" label, please correct. |
This PR fixes h5ad metadata problem.
The metadata of h5ad is not shown correctly in the history:
This little change fixes that :)