Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support configurable paths for pre-loaded digest files #151

Open
alyssadai opened this issue May 31, 2024 · 1 comment
Open

Support configurable paths for pre-loaded digest files #151

alyssadai opened this issue May 31, 2024 · 1 comment
Labels
flag:schedule Flag issue that should go on the roadmap or backlog. _flag:stale [BOT ONLY] Flag issue that hasn't been updated in a while and needs to be triaged again

Comments

@alyssadai
Copy link
Collaborator

alyssadai commented May 31, 2024

At the moment, we support pre-loaded imaging and phenotypic digest files for QPN. However, currently the app hardcodes the path to the QPN data, meaning if the name, URL, structure, etc. of the parent data repo changes, we need to modify the code so the app can find the digest files again.

digest/digest/utility.py

Lines 28 to 46 in 3654b77

# TODO:
# Could also use URLs for "imaging" or "phenotypic" locations if fetching from a remote repo doesn't slow things down too much.
# Note that this would only work for public repos or private repos with a token.
# TODO: move this to a config file?
PUBLIC_DIGEST_FILE_PATHS = {
"qpn": {
"name": "Quebec Parkinson Network",
"imaging": Path(__file__).absolute().parents[2]
/ "nipoppy-qpn"
/ "nipoppy"
/ "digest"
/ "qpn_imaging_availability_digest.csv",
"phenotypic": Path(__file__).absolute().parents[2]
/ "nipoppy-qpn"
/ "nipoppy"
/ "digest"
/ "qpn_tabular_availability_digest.csv",
}
}

To make this more maintainable and extendable (e.g., a user running their own digest instance wants to supply their own pre-loaded digest files), we could factor out the above section into a JSON file that a user can modify. With this change, we would then only need to require the user to provide a single directory path in which all their digest files live. This user-specified directory would be mounted into a consistent path inside the container. Then, in the JSON file, the user need only to specify the relative path to each digest of interest.

For the mounting portion, one option could be to create a named volume ahead of time, and then either manually mount it every time the container starts - or to just add a docker-compose file for this one container and define the volume in there.

Originally posted by @surchs in #150 (comment)

Example:

# get QPN data
git clone https://github.com/neurodatascience/nipoppy-qpn.git
# Make (initially empty QPN volume)
docker volume create qpn_data
# use silly-container just to copy data into the volume
docker run --rm -v qpn_data:/target -v $(PWD)/nipoppy-qpn:/source alpine cp -a /source/. /target/
# Use the volume as read-only access
docker run -d -p 8050:8050 -v qpn_data:/app/nipoppy-qpn neurobagel/digest:nightly
Copy link

We want to keep our issues up to date and active. This issue hasn't seen any activity in the last 75 days.
We have applied the _flag:stale label to indicate that this issue should be reviewed again.
When you review, please reread the spec and then apply one of these three options:

  • prioritize: apply the flag:schedule label to suggest moving this issue into the backlog now
  • close: if the issue is no longer relevant, explain why (give others a chance to reply) and then close.
  • archive: sometimes an issue has important information or ideas but we won't work on it soon. In this case
    apply the someday label to show that this won't be prioritized. The stalebot will ignore issues with this
    label in the future. Use sparingly!

@github-actions github-actions bot added the _flag:stale [BOT ONLY] Flag issue that hasn't been updated in a while and needs to be triaged again label Aug 16, 2024
@alyssadai alyssadai changed the title Create named volume for available digest data Make pre-loaded digest files configurable Jan 15, 2025
@alyssadai alyssadai changed the title Make pre-loaded digest files configurable Support configurable paths for pre-loaded digest files Jan 15, 2025
@alyssadai alyssadai added the flag:schedule Flag issue that should go on the roadmap or backlog. label Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flag:schedule Flag issue that should go on the roadmap or backlog. _flag:stale [BOT ONLY] Flag issue that hasn't been updated in a while and needs to be triaged again
Projects
Status: No status
Development

No branches or pull requests

1 participant