Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set default data format with enviroment variable #958

Merged
merged 9 commits into from
Oct 25, 2023
Merged

Set default data format with enviroment variable #958

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b1da892
Set default data format with enviroment variable
sm1lla Oct 24, 2023
0db11cd
format
sm1lla Oct 24, 2023
056fb03
Add env variable for chunks per shard
sm1lla Oct 24, 2023
6d3537a
Parse Vec3Int from string
sm1lla Oct 25, 2023
225d1cd
parse vec3int with regex
sm1lla Oct 25, 2023
95570fe
add type annotation
sm1lla Oct 25, 2023
a994d99
Update changelog
sm1lla Oct 25, 2023
8e28eb6
fix linting error
sm1lla Oct 25, 2023
0238b72
Merge branch 'master' into set_dataformat_zarr
normanrz Oct 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions webknossos/Changelog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ For upgrade instructions, please check the respective _Breaking Changes_ section
### Breaking Changes

### Added
- The `DEFAULT_DATA_FORMAT` and `DEFAULT_CHUNKS_PER_SHARD` can now be set with the env variables `WK_DEFAULT_DATA_FORMAT` and `WK_DEFAULT_CHUNKS_PER_SHARD`
- A `Vec3Int` can now be initialized with a string containing an int or a tuple.

### Changed
- Upgrades mypy to 1.6. [#956](https://github.com/scalableminds/webknossos-libs/pull/956)
Expand Down
14 changes: 12 additions & 2 deletions webknossos/webknossos/dataset/defaults.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,23 @@
import os

from ..geometry import Vec3Int
from .data_format import DataFormat

DEFAULT_WKW_FILE_LEN = 32
DEFAULT_CHUNK_SHAPE = Vec3Int.full(32)
DEFAULT_CHUNKS_PER_SHARD = Vec3Int.full(32)
DEFAULT_DATA_FORMAT = (
DataFormat(os.environ["WK_DEFAULT_DATA_FORMAT"])
if "WK_DEFAULT_DATA_FORMAT" in os.environ
else DataFormat.WKW
)
DEFAULT_CHUNKS_PER_SHARD = (
Vec3Int.from_str(os.environ["WK_DEFAULT_CHUNKS_PER_SHARD"])
if "WK_DEFAULT_CHUNKS_PER_SHARD" in os.environ
else Vec3Int.full(32)
)
DEFAULT_CHUNKS_PER_SHARD_ZARR = Vec3Int.full(1)
DEFAULT_CHUNKS_PER_SHARD_FROM_IMAGES = Vec3Int(128, 128, 1)
DEFAULT_BIT_DEPTH = 8
DEFAULT_DATA_FORMAT = DataFormat.WKW
PROPERTIES_FILE_NAME = "datasource-properties.json"
ZGROUP_FILE_NAME = ".zgroup"
ZATTRS_FILE_NAME = ".zattrs"
Expand Down
8 changes: 8 additions & 0 deletions webknossos/webknossos/geometry/vec3_int.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import re
from operator import add, floordiv, mod, mul, sub
from typing import Any, Callable, Iterable, List, Optional, Tuple, Union, cast

Expand Down Expand Up @@ -72,6 +73,13 @@ def from_vec_or_int(vec_or_int: Union["Vec3IntLike", int]) -> "Vec3Int":
else:
return Vec3Int(vec_or_int)

@staticmethod
def from_str(string: str) -> "Vec3Int":
if re.match(r"\(\d+,\d+,\d+\)", string):
return Vec3Int(tuple(map(int, re.findall(r"\d+", string))))
else:
return Vec3Int.full(int(string))

@property
def x(self) -> int:
return self[0]
Expand Down