Skip to content

Commit

Permalink
Breakout Zarr v2 files from #125 (#136)
Browse files Browse the repository at this point in the history
First half of #125.

Supersedes #113. Supersedes #117.
Closes #112. Closes #114. Closes #115.

## Changes

### Added

- Ship debug libs for C-Blosc on Linux and Mac.

### Changed

- Upgrades C-Blosc from v1.21.4 to v1.21.5.

### Fixed

- A bug where enabling multiscale without specifying the tile size would
cause an error.
- Exceptions thrown off the main thread are now caught and logged, and
Zarr throws an error in `append`.
  • Loading branch information
aliddell authored Oct 26, 2023
1 parent 79851b2 commit cccbda7
Show file tree
Hide file tree
Showing 43 changed files with 1,894 additions and 2,303 deletions.
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## Unreleased

### Added

- Ship debug libs for C-Blosc on Linux and Mac.

### Changed

- Upgrades C-Blosc from v1.21.4 to v1.21.5.

### Fixed

- A bug where enabling multiscale without specifying the tile size would cause an error.
- Exceptions thrown off the main thread are now caught and logged, and Zarr throws an error in `append`.

## [0.1.4](https://github.com/acquire-project/acquire-driver-zarr/compare/v0.1.3...v0.1.4) - 2023-08-11

Expand Down
4 changes: 2 additions & 2 deletions src/3rdParty/cblosc/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ if(WIN32)
set(libd "${pwd}/lib/win64/libblosc-debug.lib")
elseif(APPLE)
set(lib "${pwd}/lib/osx/libblosc.a")
set(libd ${lib})
set(libd "${pwd}/lib/osx/libblosc-debug.a")
elseif(LINUX)
set(lib "${pwd}/lib/linux-amd64/libblosc.a")
set(libd ${lib})
set(libd "${pwd}/lib/linux-amd64/libblosc-debug.a")
endif()

message(STATUS "C-Blosc: ${pwd}")
Expand Down
Binary file not shown.
Binary file modified src/3rdParty/cblosc/lib/linux-amd64/libblosc.a
Binary file not shown.
2 changes: 1 addition & 1 deletion src/3rdParty/cblosc/lib/osx/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The library here was built as follows.

Against v1.21.4 (2c2f9bd) of the c-blosc source hosted at:
Against v1.21.5 (d306135) of the c-blosc source hosted at:
https://github.com/Blosc/c-blosc

It's a universal binary compiled for `arm64` and `x86_64`.
Expand Down
Binary file added src/3rdParty/cblosc/lib/osx/libblosc-debug.a
Binary file not shown.
Binary file modified src/3rdParty/cblosc/lib/osx/libblosc.a
Binary file not shown.
Binary file modified src/3rdParty/cblosc/lib/win64/libblosc-debug.lib
Binary file not shown.
Binary file modified src/3rdParty/cblosc/lib/win64/libblosc.lib
Binary file not shown.
Binary file modified src/3rdParty/cblosc/lib/win64/libblosc.pdb
Binary file not shown.
28 changes: 13 additions & 15 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,26 +1,24 @@
add_subdirectory(3rdParty)

if(NOT TARGET acquire-core-logger)
add_subdirectory(acquire-core-libs)
endif()
if (NOT TARGET acquire-core-logger)
add_subdirectory(acquire-core-libs)
endif ()

set(tgt acquire-driver-zarr)
add_library(${tgt} MODULE
prelude.h
tiled.frame.hh
tiled.frame.cpp
chunk.writer.hh
chunk.writer.cpp
frame.scaler.hh
frame.scaler.cpp
common.hh
common.cpp
writers/writer.hh
writers/writer.cpp
writers/chunk.writer.hh
writers/chunk.writer.cpp
writers/blosc.compressor.hh
writers/blosc.compressor.cpp
zarr.hh
zarr.cpp
zarr.encoder.hh
zarr.encoder.cpp
zarr.raw.hh
zarr.raw.cpp
zarr.blosc.hh
zarr.blosc.cpp
zarr.v2.hh
zarr.v2.cpp
zarr.driver.c
)
target_enable_simd(${tgt})
Expand Down
42 changes: 42 additions & 0 deletions src/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# The Zarr Storage device

## Components

### The `StorageInterface` class.

Defines the interface that all Acquire `Storage` devices must implement, namely

- `set`: Set the storage properties.
- `get`: Get the storage properties.
- `get_meta`: Get metadata for the storage properties.
- `start`: Signal to the `Storage` device that it should start accepting frames.
- `stop`: Signal to the `Storage` device that it should stop accepting frames.
- `append`: Write incoming frames to the filesystem or other storage layer.
- `reserve_image_shape`: Set the image shape for allocating chunk writers.

### The `Zarr` class

An abstract class that implements the `StorageInterface`.
Zarr is "[a file storage format for chunked, compressed, N-dimensional arrays based on an open-source specification.](https://zarr.readthedocs.io/en/stable/index.html)"

### The `ZarrV2` class

Subclass of the `Zarr` class.
Implements abstract methods for writer allocation and metadata.
Specifically, `ZarrV2` allocates one writer of type `ChunkWriter` for each multiscale level-of-detail
and writes metadata in the format specified by the [Zarr V2 spec](https://zarr.readthedocs.io/en/stable/spec/v2.html).

### The `Writer` class

An abstract class that writes frames to the filesystem or other storage layer.
In general, frames are chunked and potentially compressed.
The `Writer` handles chunking, chunk compression, and writing.

### The `ChunkWriter` class

Subclass of the `Writer` class.
Implements abstract methods relating to writing and flushing chunk buffers.

### The `BloscCompressionParams` struct

Stores parameters for compression using [C-Blosc](https://github.com/Blosc/c-blosc).
264 changes: 0 additions & 264 deletions src/chunk.writer.cpp

This file was deleted.

Loading

0 comments on commit cccbda7

Please sign in to comment.