This repo contains client library code for accessing DataGemma, an open model that helps address the challenges of hallucination by grounding LLMs in the vast, real-world statistical data of Google's Data Commons.
There are two methodologies used to achieve this: Retrieval Interleaved Generation (RIG) and Retrieval Augmented Generation (RAG). More details can be found in the paper and blog post.
The finetuned DataGemma models are hosted in HuggingFace (RIG, RAG) and Kaggle (RIG, RAG).
To install the library, run:
pip install git+https://github.com/datacommonsorg/llm-toolsFor examples of using this library, see our Colab notebooks for RIG and RAG.
You're accessing a very early version of DataGemma. It is meant for trusted tester use (primarily for academic and research use) and not yet ready for commercial or general public use. This version was trained on a very small corpus of examples and may exhibit unintended, and at times controversial or inflammatory behavior. Please anticipate errors and limitations as we actively develop this large language model interface.
Your feedback and evaluations are critical to refining DataGemma's performance and will directly contribute to its training process. Known limitations are detailed in the paper, and we encourage you to consult it for a comprehensive understanding of DataGemma's current capabilities.