Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix metadata setting for empty tiff files #18533

Closed

Conversation

bernt-matthias
Copy link
Contributor

had a tool that created (by error) an empty tiff file:

galaxy.metadata DEBUG 2024-07-11 09:42:55,503 [pN:main.3,p:97809,tN:SlurmRunner.work_thread-2] setting metadata externally failed for HistoryDatasetAssociation 903743: Traceback (most recent call last):
  File "/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4053, in __init__
    byteorder = {b'II': '<', b'MM': '>', b'EP': '<'}[header[:2]]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: b''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/metadata/set_metadata.py", line 518, in set_metadata_portable
    set_meta(dataset, file_dict)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/metadata/set_metadata.py", line 205, in set_meta
    set_meta_with_tool_provided(
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/metadata/set_metadata.py", line 141, in set_meta_with_tool_provided
    dataset_instance.datatype.set_meta(dataset_instance, **set_meta_kwds)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/datatypes/images.py", line 113, in set_meta
    with tifffile.TiffFile(dataset.get_file_name()) as tif:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4055, in __init__
    raise TiffFileError(f'not a TIFF file {header!r}') from exc
tifffile.tifffile.TiffFileError: not a TIFF file b''

galaxy.jobs.runners ERROR 2024-07-11 09:42:55,509 [pN:main.3,p:97809,tN:SlurmRunner.work_thread-2] (261108/23547903) Job wrapper finish method failed
Traceback (most recent call last):
  File "/gpfs1/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4053, in __init__
    byteorder = {b'II': '<', b'MM': '>', b'EP': '<'}[header[:2]]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: b''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/runners/__init__.py", line 656, in _finish_or_resubmit_job
    job_wrapper.finish(
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/__init__.py", line 1967, in finish
    self._finish_dataset(output_name, dataset, job, context, final_job_state, remote_metadata_directory)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/__init__.py", line 1778, in _finish_dataset
    dataset.datatype.set_meta(dataset, overwrite=False)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/datatypes/images.py", line 113, in set_meta
    with tifffile.TiffFile(dataset.get_file_name()) as tif:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs1/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4055, in __init__
    raise TiffFileError(f'not a TIFF file {header!r}') from exc
tifffile.tifffile.TiffFileError: not a TIFF file b''

(Please replace this header with a description of your pull request. Please include BOTH what you did and why you made the changes. The "why" may simply be citing a relevant Galaxy issue.)
(If fixing a bug, please add any relevant error or traceback)
(For UI components, it is recommended to include screenshots or screencasts)

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

had a tool that created (by error) an empty tiff file:

```
galaxy.metadata DEBUG 2024-07-11 09:42:55,503 [pN:main.3,p:97809,tN:SlurmRunner.work_thread-2] setting metadata externally failed for HistoryDatasetAssociation 903743: Traceback (most recent call last):
  File "/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4053, in __init__
    byteorder = {b'II': '<', b'MM': '>', b'EP': '<'}[header[:2]]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: b''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/metadata/set_metadata.py", line 518, in set_metadata_portable
    set_meta(dataset, file_dict)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/metadata/set_metadata.py", line 205, in set_meta
    set_meta_with_tool_provided(
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/metadata/set_metadata.py", line 141, in set_meta_with_tool_provided
    dataset_instance.datatype.set_meta(dataset_instance, **set_meta_kwds)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/datatypes/images.py", line 113, in set_meta
    with tifffile.TiffFile(dataset.get_file_name()) as tif:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4055, in __init__
    raise TiffFileError(f'not a TIFF file {header!r}') from exc
tifffile.tifffile.TiffFileError: not a TIFF file b''

galaxy.jobs.runners ERROR 2024-07-11 09:42:55,509 [pN:main.3,p:97809,tN:SlurmRunner.work_thread-2] (261108/23547903) Job wrapper finish method failed
Traceback (most recent call last):
  File "/gpfs1/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4053, in __init__
    byteorder = {b'II': '<', b'MM': '>', b'EP': '<'}[header[:2]]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: b''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/runners/__init__.py", line 656, in _finish_or_resubmit_job
    job_wrapper.finish(
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/__init__.py", line 1967, in finish
    self._finish_dataset(output_name, dataset, job, context, final_job_state, remote_metadata_directory)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/__init__.py", line 1778, in _finish_dataset
    dataset.datatype.set_meta(dataset, overwrite=False)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/datatypes/images.py", line 113, in set_meta
    with tifffile.TiffFile(dataset.get_file_name()) as tif:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs1/data/galaxy_server/galaxy/.venv/lib/python3.11/site-packages/tifffile/tifffile.py", line 4055, in __init__
    raise TiffFileError(f'not a TIFF file {header!r}') from exc
tifffile.tifffile.TiffFileError: not a TIFF file b''
```
@github-actions github-actions bot added this to the 24.2 milestone Jul 11, 2024
@bernt-matthias bernt-matthias changed the title fix metadata setting for empty tiff files Fix metadata setting for empty tiff files Jul 11, 2024
@mvdbeek
Copy link
Member

mvdbeek commented Jul 11, 2024

Isn't the error and resulting failed metadata state what you need to know that there is an error and that this file is not in fact a tiff file ? We've got quite some precedence in that for other datatypes, e.g. bam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants