Guarded RAG Assistant

The guarded RAG assistant is an easily customizable recipe for building a RAG-powered chatbot.

In addition to creating a hosted, shareable user interface, the guarded RAG assistant provides:

• Business logic and LLM-based guardrails.
• A predictive secondary model that evaluates response quality.
• GenAI-focused custom metrics.
• DataRobot MLOps hosting, monitoring, and governing the individual back-end deployments.

Warning

Application templates are intended to be starting points that provide guidance on how to develop, serve, and maintain AI applications. They require a developer or data scientist to adapt and modify them for their business requirements before being put into production.

Table of contents

1. Setup
2. Architecture overview
3. Why build AI Apps with DataRobot app templates?
4. Make changes
   • Change the RAG documents
   • Change the LLM
   • Change the RAG prompt
   • Custom front-end
   • Custom RAG logic
5. Share results
6. Delete all provisioned resources
7. Setup for advanced users
8. Data Privacy

Setup

Important

If you are running this template in a DataRobot codespace, pulumi is already configured and the repo is automatically cloned; skip to Step 3.

1. If pulumi is not already installed, install the CLI following instructions here. After installing for the first time, restart your terminal and run:

   pulumi login --local  # omit --local to use Pulumi Cloud (requires separate account)

2. Clone the template repository.

   git clone https://github.com/datarobot-community/guarded-rag-assistant.git
   cd guarded-rag-assistant

3. Rename the file .env.template to .env in the root directory of the repo and populate your credentials. This template is pre-configured to use an Azure OpenAI endpoint. If you wish to use a different LLM provider, modifications to the code will be necessary.

   Please refer to the documentation inside .env.template

4. In a terminal, run:

   python quickstart.py YOUR_PROJECT_NAME  # Windows users may have to use `py` instead of `python`

   Python 3.9+ is required.

Advanced users desiring control over virtual environment creation, dependency installation, environment variable setup, and pulumi invocation see the advanced setup instructions.

Architecture overview

App templates contain three families of complementary logic. For Guarded RAG you can opt-in to fully custom RAG logic and a fully custom frontend or utilize DR's off the shelf offerings:

• AI logic: Necessary to service AI requests and produce predictions and completions.

  deployment_*/  # Predictive model scoring logic, RAG completion logic (DIY RAG)
  notebooks/  # Document chunking, VDB creation logic (DIY RAG)
  

• App Logic: Necessary for user consumption; whether via a hosted front-end or integrating into an external consumption layer.

  frontend/  # Streamlit frontend (DIY frontend)
  docsassist/  # App business logic & runtime helpers (DIY front-end)
  

• Operational Logic: Necessary to activate DataRobot assets.

  __main__.py  # Pulumi program for configuring DataRobot to serve and monitor AI and App logic
  infra/  # Settings for resources and assets created in DataRobot
  

Why build AI Apps with DataRobot app templates?

App Templates transform your AI projects from notebooks to production-ready applications. Too often, getting models into production means rewriting code, juggling credentials, and coordinating with multiple tools & teams just to make simple changes. DataRobot's composable AI apps framework eliminates these bottlenecks, letting you spend more time experimenting with your ML and app logic and less time wrestling with plumbing and deployment.

• Start building in minutes: Deploy complete AI applications instantly, then customize the AI logic or the front-end independently (no architectural rewrites needed).
• Keep working your way: Data scientists keep working in notebooks, developers in IDEs, and configs stay isolated. Update any piece without breaking others.
• Iterate with confidence: Make changes locally and deploy with confidence. Spend less time writing and troubleshooting plumbing and more time improving your app.

Each template provides an end-to-end AI architecture, from raw inputs to deployed application, while remaining highly customizable for specific business requirements.

Make changes

Change the RAG documents

1. Replace assets/datarobot_english_documentation_docsassist.zip with a new zip file containing .pdf, .docx, .md, or .txt documents (example alternative docs here).
2. Update the rag_documents setting in infra/settings_main.py to specify the local path to the new zip file.
3. Run pulumi up to update your stack.

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

Change the LLM

1. Modify the LLM setting in infra/settings_generative.py by changing LLM=GlobalLLM.AZURE_OPENAI_GPT_4_O to any other LLM from the GlobalLLM object.
2. Provide the required credentials in .env dependent on your choice.
3. Run pulumi up to update your stack (Or rerun your quickstart).

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

Change the RAG prompt

1. Modify the system_prompt variable in infra/settings_generative.py with your desired prompt.
2. If using fully custom RAG logic, instead please change the stuff_prompt variable in notebooks/build_rag.ipynb.

Fully custom front-end

1. Edit infra/settings_main.py and update application_type to ApplicationType.DIY
   • Optionally, update APP_LOCALE in docsassist/i18n.py to toggle the language. Supported locales are Japanese and English, with English set as the default.
2. Run pulumi up to update your stack with the example custom Streamlit frontend:

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

3. After provisioning the stack at least once, you can also edit and test the Streamlit front-end locally using streamlit run app.py from the frontend/ directory (don't forget to initialize your environment using set_env).

   source set_env.sh  # On windows use `set_env.bat`
   cd frontend
   streamlit run app.py

Fully custom RAG chunking, vectorization, and retrieval

1. Install additional requirements (e.g. FAISS, HuggingFace).

   source set_env.sh  # On windows use `set_env.bat`
   pip install -r requirements-extra.txt

2. Edit infra/settings_main.py and update rag_type to RAGType.DIY.
3. Run pulumi up to update your stack with the example custom RAG logic.

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

4. Edit notebooks/build_rag.ipynb to customize the doc chunking, vectorization logic.
5. Edit deployment_diy_rag/custom.py to customize the retrieval logic & LLM call.
6. Run pulumi up to update your stack.

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

Share results

1. Log into the DataRobot application.
2. Navigate to Registry > Applications.
3. Navigate to the application you want to share, open the actions menu, and select Share from the dropdown.

Delete all provisioned resources

pulumi down

Setup for advanced users

For manual control over the setup process adapt the following steps for MacOS/Linux to your environent:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
source set_env.sh
pulumi stack init YOUR_PROJECT_NAME
pulumi up 

e.g. for Windows/conda/cmd.exe this would be:

conda create --prefix .venv pip
conda activate .\.venv
pip install -r requirements.txt
set_env.bat
pulumi stack init YOUR_PROJECT_NAME
pulumi up 

For projects that will be maintained, DataRobot recommends forking the repo so upstream fixes and improvements can be merged in the future.

Data Privacy

Your data privacy is important to us. Data handling is governed by the DataRobot Privacy Policy, please review before using your own data with DataRobot.