-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make data source definition more flexible to allow for dataset updates #37
Comments
I agree with you that this is urgently needed. This will probably require a bit more thought and some more involved restructuring/refactoring. Would you have a proposal on how to go about this and/or resources to support the implementation of such a proposal? |
Honestly, my R knowledge is too limited to be of technical help here. Design-wise, I believe that updating existing data sources might be tricky if you want to keep backward compatibility. |
@mlondschien I believe we briefly discussed this offline, but just to re-iterate: what you claim above is not true.
Data source definitions (as are concept definitions) are user configurable. Maybe the docs at |
Thanks @nbenn! Some guidance on how to do this, optimally an example, would be great. |
I don't know in what ways newer mimic versions differ from what we had ~3 years ago. I'm assuming it's mostly adding some table(s), adding/removing patients. Maybe they dropped CareVue patients? That would simplify concept specification as fewer items have to be considered, but again seems largely backwards compatible. From a distance, I'm hoping that these changes do not really require changing mimic-specific logic but can mostly be handled by config. Maybe also check in with @prockenschaub as he's working on adding new mimic to ricu I believe. |
I created a (temporary) repository that collects the all the different databases configs, including a short description of how to load them. @mlondschien maybe you can have a look and check that it works for you. Long-term, I think it would be nice for backward compatibility to include those earlier versions with @nbenn while I agree that what Malte wants should already be possible, I think we need to provid more guidance for the user. From my experience, the learning curve of |
I have discussed this with @nbenn. While I can see that having dataset versioning would be useful, this would create quite a lot of overhead in terms of re-structuring things. Therefore, we will not prioritize this currently, given the fact that other things have a better value/time invested ratio. Instead, we have opted for a different solution: we will try to be up-to-date with the "latest" version of each dataset. For cases in which an older version of a dataset is needed, one may add this dataset locally. If you think examples for how this could be done would be helpful, please feel free to open a new issue. Adding local datasets is described to some degree here. |
Related: #33, #28.
The links (and thus versions) of data sources are currently hardcoded in the data-sources.json. This makes
ricu
unnecessarily inflexible. Adding a new data source requires changing thericu
source code (e.g., #30) as does benefitting from the data source's updates (e.g.,caregiver_id
was added to miiv tables in v2.2, which is currently not accessible viaricu
).The text was updated successfully, but these errors were encountered: