Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find all datasets that share a layer #6726

Open
fm3 opened this issue Jan 6, 2023 · 2 comments
Open

Find all datasets that share a layer #6726

fm3 opened this issue Jan 6, 2023 · 2 comments

Comments

@fm3
Copy link
Member

fm3 commented Jan 6, 2023

Many datasets share a layer (e.g. one color layer, many segmentation layers)
wk should detect this (by resolving symlinks or the paths in datasource-properties.json) and store this information in postgres so that queries can be possible against this info (e.g. list all datasets with this layer)

@fm3
Copy link
Member Author

fm3 commented Nov 15, 2024

This info could also be used when deleting datasets. If the layer is still referenced by another dataset (e.g. created via compose), the data should be moved, and other references/symlinks changed

@fm3
Copy link
Member Author

fm3 commented Jan 12, 2025

@frcroth I don’t have a concrete design in mind for this yet. I guess we could store some property like “realpath” with the layers (or mags?) in postgres. We’d have to create a way for that information to get there. We’d have to deal with remote dataset layers.
The main use case right now is deletion #8279 – so remote datasets may not be the most important part right now.
I can show you around the existing code on dataset reporting from the datastore to postgres. Maybe you could then consider some ideas for how to solve this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants