Project Overiew

This project is an API-based RAG system designed to facilitate communication with text-based PDFs, Word documents, and text files. It leverages Language Model (LLMs) and Embeddings that run locally, ensuring user privacy and control.

Architecture

Requirements

All Open Sourced LLMs , Vector DB , Models and Tools are Genereally prefered and have used Docker Compose that will start the application to easily

Components

• .Net API
• Qdrant (Vector Database)
• Hugging Face Embeddings
• Ollama (LLM Provider)

Start the application

Before starting the application, need to decide which llm , vector model a person want.

`docker compose up -d`

Start LLM

In Application their are major 2 ways to use the LLM that is LMStudio and OLLama and both are configured in appsettigs.json file that can make it easy to use and with no other cofiguration needed.

LM Studio

We can run using its own application and then start its server using the application.

Ollama

We can directly use the Ollama that can start using the docker compose easilt and then using the command below.

docker exec -it ollama ollama pull <model name>

Start Embedding Model

The Embedding model by default is setup but can changed in the docker compose of the model name.

Start Qdrant and .Net API

By running the docker compose one can start the application easily.

Use The APPLICATION

Using Docker Compose a person can start the application after doing all the neccesarry steps.

Use the Below curls to use the application various steps

CURLS

Get Token For File Sending

curl --location 'http://localhost:3255/antiforgery/token'

Send File To The RAG

curl --location 'http://localhost:3255/api/file?collection=<CollectionName>' \
--header 'X-XSRF-TOKEN:<Token> ' \
--form '=@"/test.pdf"' 

Chat With DATA

curl --location 'http://0.0.0.0:3255/api/chat' \
--header 'accept: */*' \
--header 'Content-Type: application/json' \
--data '{
  "chatId": "6abdc",
  "collectionName": "rustdocs",
  "userQuery": "what is cargo"
}' 

Note

Can increase or decrease the Embeddings volume using the Search Paramaeters by changing the limit or the Releveance of them.

The Docker Compose are CPU only to allow GPU need to Use the Docker Files and Configuration of GPUs

Example

Question : what is Cargo In Rust

Normal LLM Answer

Cargo in Rust refers to the process of transferring data, such as structured or unstructured data, between different processing components. Cargo can be either transmitted using network protocols like HTTP, TCP/IP or UDP, or stored as binary files in a database like PostgreSQL, MySQL, or MongoDB. Cargo is typically used for moving large amounts of data from one system to another or passing information between various parts of an application. In Rust, it can also be used for streaming data, such as real-time data feeds or machine learning datasets.

RAG Application Answer

Cargo is RusT's build system and package manager. It automates the process of building, managing, and distributing packages (i.e., binaries or libraries) in a project that uses RusT. This helps to organize and manage projects using RusT by creating a top-level project directory, containing source code files, README files, license information, configuration files, and anything else that is not related to the project's code. By using Cargo, developers can easily organize their projects into packages that provide functionality, and convert existing projects to use Cargo instead of building them manually.

In this Sample have used TinnyLLama with Rust Playbook.