You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Oct 1, 2024. It is now read-only.
Getting arbitrary compression options across the pyo3 interface seems like it could be a pain. Also, choosing an API may require a bit of thought:
The h5py way
Takes a compression: Optional[str]=None, and a compression_opts: Optional[Any]=None. For level-5 gzip, these would be "gzip" and 5. h5py doesn't support that many compression types, but n5 may want to support things which fairly complicated configurations, as the config is just stored as a JSON object. Zarr supports compression layers. Splitting across multiple arguments is a bit unfortunate.
the z5 way
**compression_opts are kwargs. This is nice because it means the type of each option can be consistent, but it means that you prevent yourself adding any other **kwargs in the future.
The strongly-typed way
As rust (and presumably java) does: take a single object whose type describes the type of compression, and whose members configure that compression. This would be discoverable and hopefully easy to map to rust (namedtuple -> struct). But it means duplicating some of the compression-configuration machinery from rust into python.
The sketchy MVP way
Whenever you create a dataset (writing its attributes.json), python then re-opens the attributes file and dumps an arbitrary compression JSON object in there, hopefully before any data is written.
The text was updated successfully, but these errors were encountered:
As mentioned in #65.
Getting arbitrary compression options across the pyo3 interface seems like it could be a pain. Also, choosing an API may require a bit of thought:
The h5py way
Takes a
compression: Optional[str]=None
, and acompression_opts: Optional[Any]=None
. For level-5 gzip, these would be"gzip"
and5
. h5py doesn't support that many compression types, but n5 may want to support things which fairly complicated configurations, as the config is just stored as a JSON object. Zarr supports compression layers. Splitting across multiple arguments is a bit unfortunate.the z5 way
**compression_opts
are kwargs. This is nice because it means the type of each option can be consistent, but it means that you prevent yourself adding any other**kwargs
in the future.The strongly-typed way
As rust (and presumably java) does: take a single object whose type describes the type of compression, and whose members configure that compression. This would be discoverable and hopefully easy to map to rust (namedtuple -> struct). But it means duplicating some of the compression-configuration machinery from rust into python.
The sketchy MVP way
Whenever you create a dataset (writing its
attributes.json
), python then re-opens the attributes file and dumps an arbitrary compression JSON object in there, hopefully before any data is written.The text was updated successfully, but these errors were encountered: