-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Converted Datadog_API.json to Datadog_API.jsonl #678
base: main
Are you sure you want to change the base?
Conversation
Hi, I was also able to convert most of the other .json files under gorilla/data/apizoo to .jsonl except for a couple with formatting issues (Gmail_API, KubernetesAPI, Kubernetes...PodTemplateAPI, Salesforce...REST Format). I have encountered similar formatting issues with the data in gorilla/bfcl/data. |
Hi @ShishirPatil @HuanzhiMao, just wanted to check on this PR, I've also been looking through the bfcl folder and specifically files importing json, wanted to confirm which files are part of the bfcl pipeline that need to be updated from json to jsonl such as berkeley-function-call-leaderboard/bfcl/eval_checker/executable_eval/executable_checker.py, berkeley-function-call-leaderboard/bfcl/eval_checker/eval_runner_helper.py, some of the models in berkeley-function-call-leaderboard/bfcl/model_handler/oss_model, etc. Thanks so much, appreciate your help with this PR. |
Only the |
Used jsonlines library to change output to a jsonl file instead of json for modified_data
Commented jsonlines changes as functions mainly use json internally so jsonlines interactions aren't needed to write to new files
Modified functions involving json files to read from and write to corresponding jsonl files via jsonlines library
This file is already a valid .jsonl as per https://jsonlines.org/validator/, so I have updated the name of the file accordingly so it can work with the updated bfcl pipeline.
Hi, @ShishirPatil @HuanzhiMao, I have updated the python files (_apply_function_credential_config.py, etc) that read and wrote json files to handle jsonl files using the jsonlines library (https://jsonlines.readthedocs.io/en/latest/), commenting out the previous code. I have also updated the files in the bfcl/data folder since they were validated as proper jsonl files by https://jsonlines.org/validator/ when I pasted their contents, so their extensions could be updated to jsonl to be consistent with the changes to the python files. Kindly review the changes in the PR, thanks so much. |
Hi,
I was able to complete this conversion in gorilla/data/apizoo locally using Jupyter Notebook and upload the jsonl file, wanted to make sure if I should follow this process for all the other .json files in the dataset as well?