MedHelm: Add VQA-RAD scenario and specs #3246
Conversation
Co-authored-by: Yifan Mai <[email protected]>
…rd-crfm#3185) Co-authored-by: Yifan Mai <[email protected]>
There was a problem hiding this comment.
Thanks, this looks good in general. However, it looks like your pull request does not pass the linter. Could you install the [dev] extras (or you can install the linter directly using pip install black==24.3.0 flake8==5.0.4 mypy==1.5.1) and then run ./pre-commit.sh, and then make the requested fixes?
| extra_data={ | ||
| "id": index, | ||
| "image_path": image_path, | ||
| }, |
There was a problem hiding this comment.
extra_data is unused and should be deleted. If you need to keep ID, you can set the id field (e.g. id=f"id{id}") on the Instance itself.
No worries, I'll work on that. Thank you for reviewing it! |
|
Also, in general it's okay to use unofficial third part mirrors of datasets on Hugging Face; we already do this for some other scenarios. |
The VQA-rad scenario is implemented using data from this huggingface dataset. My question regarding this is whether we should instead point to where the data was originally published instead: Open Science Framework Homepage
The PR was tested with
HuggingFaceM4/idefics2-8b:Besides the scenario and specs, this PR includes an addition to the documentation, explaining how to get helm running on cloud computes that have network mounted directories, which leads to
sqlite3.OperationalError: database is locked.