Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set default data format with enviroment variable #958

Merged
merged 9 commits into from
Oct 25, 2023
Merged

Set default data format with enviroment variable #958

Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b1da892
Set default data format with enviroment variable
sm1lla Oct 24, 2023
0db11cd
format
sm1lla Oct 24, 2023
056fb03
Add env variable for chunks per shard
sm1lla Oct 24, 2023
6d3537a
Parse Vec3Int from string
sm1lla Oct 25, 2023
225d1cd
parse vec3int with regex
sm1lla Oct 25, 2023
95570fe
add type annotation
sm1lla Oct 25, 2023
a994d99
Update changelog
sm1lla Oct 25, 2023
8e28eb6
fix linting error
sm1lla Oct 25, 2023
0238b72
Merge branch 'master' into set_dataformat_zarr
normanrz Oct 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions webknossos/webknossos/dataset/defaults.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
import os

from ..geometry import Vec3Int
from .data_format import DataFormat

DEFAULT_WKW_FILE_LEN = 32
DEFAULT_CHUNK_SHAPE = Vec3Int.full(32)
DEFAULT_CHUNKS_PER_SHARD = Vec3Int.full(32)
DEFAULT_DATA_FORMAT = (
DataFormat.WKW if not os.environ.get("DATA_FORMAT") == "zarr3" else DataFormat.Zarr3
sm1lla marked this conversation as resolved.
Show resolved Hide resolved
)
DEFAULT_CHUNKS_PER_SHARD = (
Vec3Int.full(32) if DEFAULT_DATA_FORMAT == DataFormat.WKW else Vec3Int.full(16)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make a separate env var WK_DEFAULT_CHUNKS_PER_SHARD for this.

)
DEFAULT_CHUNKS_PER_SHARD_ZARR = Vec3Int.full(1)
DEFAULT_CHUNKS_PER_SHARD_FROM_IMAGES = Vec3Int(128, 128, 1)
DEFAULT_BIT_DEPTH = 8
DEFAULT_DATA_FORMAT = DataFormat.WKW
PROPERTIES_FILE_NAME = "datasource-properties.json"
ZGROUP_FILE_NAME = ".zgroup"
ZATTRS_FILE_NAME = ".zattrs"
Expand Down
Loading