Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[23.2] Npz sniffing: do not read the whole file #17674

Merged
merged 1 commit into from
Mar 12, 2024

Conversation

bernt-matthias
Copy link
Contributor

Backport of #17672

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@github-actions github-actions bot added this to the 23.2 milestone Mar 12, 2024
@mvdbeek mvdbeek merged commit 93eb541 into galaxyproject:release_23.2 Mar 12, 2024
86 of 99 checks passed
Copy link

This PR was merged without a "kind/" label, please correct.

@bernt-matthias bernt-matthias deleted the topic/npz-sniff branch March 12, 2024 14:38
@nsoranzo
Copy link
Member

@bernt-matthias This (as well as #17672 ) broke some integration tests, can you check please?

@mvdbeek
Copy link
Member

mvdbeek commented Mar 12, 2024

I think I found a nice way to fix the sniffer by checking if any zipped file is a .npy file in #17679

@bernt-matthias
Copy link
Contributor Author

bernt-matthias commented Mar 12, 2024

Hrm. Seems that npz files are just zip files .. contrary to the docs the only property that distinguishes them from zip files numpy/numpy#26004 is not enforced.

I could implement this test, i.e. get a file listing from the zip and check if all are .npy. Otherwise I would suggest to drop the sniffer.

@mvdbeek
Copy link
Member

mvdbeek commented Mar 12, 2024

That's what #17679 does

@mvdbeek
Copy link
Member

mvdbeek commented Mar 12, 2024

except it's valid for some files not to be .npy files, so I'm using any() ... I don't think we care since sniffing doesn't have to be perect

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants