This is an Acquire Driver that supports chunked streaming to zarr.
- Zarr
- ZarrBlosc1ZstdByteShuffle
- ZarrBlosc1Lz4ByteShuffle
- ZarrV3
- ZarrV3Blosc1ZstdByteShuffle
- ZarrV3Blosc1Lz4ByteShuffle
Zarr has additional capabilities relative to the basic storage devices, namely chunking, compression, and multiscale storage.
To compress while streaming, you can use one of the ZarrBlosc1*
devices.
Chunking is configured using storage_properties_set_chunking_props()
when configuring your video stream.
Multiscale storage can be enabled or disabled by calling storage_properties_set_enable_multiscale()
when configuring
the video stream.
For the Zarr v3 version of each device, you can use the ZarrV3*
devices.
Note: Zarr v3 is not yet supported by the Python OME-Zarr library, so you
will not be able to read multiscale metadata from the resulting dataset.
You can configure chunking by calling storage_properties_set_chunking_props()
on your StorageProperties
object
after calling storage_properties_init()
.
There are 4 parameters you can set to determine the chunk size, namely tile_width
, tile_height
, tile_planes
,
and bytes_per_chunk
:
int
storage_properties_set_chunking_props(struct StorageProperties* out,
uint32_t tile_width,
uint32_t tile_height,
uint32_t tile_planes,
uint64_t max_bytes_per_chunk)
A collection of frames. |
A tile is a contiguous section, or region of interest, of a frame.
A collection of frames, divided into tiles. |
A chunk is nothing more than some number of stacked tiles from subsequent frames, with each tile in a chunk having the same ROI in its respective frame.
A collection of frames, divided into tiles. A single chunk has been highlighted in red. |
You can specify the width and height, in pixels, of each tile, and if your frame size has more than one plane, you can specify the number of planes you want per tile as well. If any of these values are unset (equivalently, set to 0), or if they are set to a value larger than the frame size, the full value of the frame size along that dimension will be used instead. You should take care that the values you select won't result in tile sizes that are too small or too large for your application.
The max_bytes_per_chunk
parameter can be used to cap the size of a chunk.
A minimum of 16 MiB is enforced, but no maximum, so if you are compressing you must ensure that you have sufficient
memory for all your chunks to be stored in memory at once.
Suppose your frame size is 1920 x 1080 x 1, with a pixel type of unsigned 8-bit integer. You can use a tile size of 640 x 360 x 1, which will divide your frame evenly into 9 tiles. You want chunk sizes of at most 64 MiB. You would configure your storage properties as follows:
storage_properties_set_chunking_props(&storage_props,
640,
360,
1,
64 * 1024 * 1024);
Note that 64 * 1024 * 1024 / (640 * 360) = 291.2711111111111, so each chunk will contain 291 tiles, or about 63.94 MiB raw, before compression.
Compression is done via Blosc. Supported codecs are lz4 and zstd, which can be used by using the ZarrBlosc1Lz4ByteShuffle and ZarrBlosc1ZstdByteShuffle devices, respectively. For a comparison of these codecs, please refer to the Blosc docs.
In order to enable or disable multiscale storage for your video stream, you can call
storage_properties_set_enable_multiscale()
on your StorageProperties
object after calling
storage_properties_init()
on it.
int
storage_properties_set_enable_multiscale(struct StorageProperties* out,
uint8_t enable);
To enable, pass 1
as the enable
parameter, and to disable, pass 0
.
If enabled, the Zarr writer will write a pyramid of frames, with each level of the pyramid halving each dimension of the previous level, until the dimensions are less than or equal to a single tile.
Suppose your frame size is 1920 x 1080, with a tile size of 384 x 216. Then the sequence of levels will have dimensions 1920 x 1080, 960 x 540, 480 x 270, and 240 x 135.