You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Aug 21, 2024. It is now read-only.
As @miyunari proposed, we should evaluate the usage of zstandard compressed files as delivery method of raw image data.
Currently the retriever creates files that are zst encoded (AWS) as well as non encoded files (google and azure).
This gives the impression that the question of encoding raw image data isn't well enough thought out.
Expected Outcome
If the decision is made to transition to zst encoding, then the transformer needs to be extended to enable decompression of said files.
Implementation example: #574
It also requires that the retriever creates consistent data across the board.
All created raw data files need to be zst encoded, not only AWS
It needs to serve a purpose
The text was updated successfully, but these errors were encountered:
As @miyunari proposed, we should evaluate the usage of zstandard compressed files as delivery method of raw image data.
Currently the retriever creates files that are zst encoded (AWS) as well as non encoded files (google and azure).
This gives the impression that the question of encoding raw image data isn't well enough thought out.
Expected Outcome
If the decision is made to transition to zst encoding, then the transformer needs to be extended to enable decompression of said files.
Implementation example: #574
It also requires that the retriever creates consistent data across the board.
The text was updated successfully, but these errors were encountered: