BOSS backend #59
BOSS backend #59
Conversation
| *(f"{l}:{r}" for l, r in zip(*range_box)), | ||
| ] | ||
| ) | ||
| async with await self.auth_get(url, token_key) as r: |
There was a problem hiding this comment.
There are multiple dataformats supported:
https://docs.theboss.io/docs/cutout-get#section-getting-data-volumes
I also tried application/blosc-python, which is simpler to implement, but uses pickle internally, which makes it slower. image/jpeg might be an option to save bandwidth, but I think raw data is fine for now. Opinions?
data/datasets.json
Outdated
| "domain": "https://api.boss.neurodata.io", | ||
| "collection": "kasthuri", | ||
| "experiment": "kasthuri11", | ||
| "username": "jstriebel", |
There was a problem hiding this comment.
@normanrz I would like to check in one or two datasets, so that there is immediately something to try out. Could you create a user on neurodata.io with scmboy for that?
There was a problem hiding this comment.
or @hotzenklotz? ^
There was a problem hiding this comment.
Hmm. I think it is reasonable to ask people to create an account when they want to use the boss integration. I wouldn't want to put credentials into the repo.
There was a problem hiding this comment.
When using it with wk, one will be asked via the frontend anyways. This one would be rather for development, where I would find it useful if we have one or two demo-datasets in the repo, that are working out-of-the box (for dev & review).
The credentials can not be used for anything, except retrieving data from neurodata.io and possibly exceeding a rate-limit, if there is one.
There was a problem hiding this comment.
I could also read credentials from an gitignored .env file as a fallback. That would make it easier to develop locally without caring about checking in personal credentials by accident or not.
There was a problem hiding this comment.
I implemented the second option now (env vars), this seems quite useful.
This PR adds image-data support for the BOSS backend.
I would recommend to review the files in this order:
The BOSS API is documented here:
https://docs.theboss.io/docs
TODO
fixes #2
follow-up issues #60, #61, #62, #63 (not all important, though)