Skip to content

Commit

Permalink
doc changes + minor pkg additions
Browse files Browse the repository at this point in the history
  • Loading branch information
raghavdixit99 committed Feb 7, 2024
1 parent 5112e13 commit 98e6de1
Show file tree
Hide file tree
Showing 4 changed files with 42 additions and 91 deletions.
90 changes: 0 additions & 90 deletions applications/GTE_mlx_RAG/CLI_example.ipynb

This file was deleted.

40 changes: 40 additions & 0 deletions applications/GTE_mlx_RAG/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@

## CLI RAG application using GTE MLX (Apple silicon) and Lancedb
This is a Command Line Interface app designed to provide users with quick and accurate responses to their queries based on the input file (pdf) via RAG architecture.

## Overview
It is built using `lancedb.embeddings.gte` embedding function which uses General Text Embeddings (GTE) model to embed documents, we have added support for Apple silicon devices by adding the MLX format of the model which can be accessed by the 'mlx=True' argument in the function. This app uses `mlx_lm` as the LLM for generating the final response.

- ingest_pdf.py - Extracts text from input pdf and stores in vectorDB.
- query_gte.py - Retrieves context relevent to question from vectorDB and augments prompt to generate RAG response.


## Getting started

* Install requirements

Please install lancedb via git instead of PyPI as some latest features might be missing, to get the latest code, run the following in your virtual env :
```bash
pip install -e "git+https://github.com/lancedb/lancedb.git#egg=lancedb&subdirectory=python"

python3 -m pip install -r requirements.txt
```

* Create vectors from a pdf file and store in lancedb

Store your input data in './data/' folder and run the following to ingest the vectors :
```bash
python3 ingest_pdf.py --pdf ./data/mossformer.pdf
```

* Query database and generate response

Run the following command by adding your query after the '--question' argument to get the response from the llm
```bash
python3 query_gte.py --question "Explain 2.2.- Hybrid MossFormer and Recurrent Modules in simpler language."
```


## Sample response

![image](../../assets/GTE.png)
3 changes: 2 additions & 1 deletion applications/GTE_mlx_RAG/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
unstructured[pdf]
mlx
mlx_lm
mlx_lm
ai2-olmo
Binary file added assets/GTE.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 98e6de1

Please sign in to comment.