diff --git a/README.md b/README.md index 090d438..291c1ff 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,8 @@
[pykoi](https://www.cambioml.com/pykoi/) is an open-source python library for improving LLMs with RLHF. We provide a unified interface including RLHF/RLAIF data and feedback collection, finetuning with reinforcement learning and reward modeling, and LLM comparisons. @@ -41,43 +42,51 @@ Reinforcement Learning with Human Feedback (RLHF) is a unique training paradigm ## Installation -To get started with `pykoi`, you can choose to one of following compute options: CPU (e.g. your laptop) or GPU (e.g. EC2). +To get started with pykoi, you can choose from any of the installation options. The choice should be based on the features you need (e.g., RAG, RLHF or all) and the compute resources you have, such as a CPU (e.g., your laptop) or GPU (e.g., AWS EC2 or SageMaker). -### Option 1: CPU (e.g. your laptop) -Installation on a CPU is simple if you have conda. If not, install [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) for your operating system. +### Option 1: RAG (CPU) +This option allows you to run RAG on a CPU using either the OpenAI API or the Anthropic Claude2 API. Installation of RAG (CPU) is simple if you have conda. If not, install [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) for your operating system. First, create a conda environment on your terminal using: ``` conda create -n pykoi python=3.10 -y -conda activate pykoi +conda activate pykoi # some OS requires `source activate pykoi` ``` -Then install `pykoi` and the compatible [pytorch based on your os](https://pytorch.org/get-started) +Then install `pykoi` and the compatible [pytorch based on your os](https://pytorch.org/get-started/locally/) ``` -pip3 install pykoi +pip3 install "pykoi[rag]" pip3 install torch ``` -### Option 2: GPU (e.g. EC2 or SageMaker) +### Option 2: RAG (GPU) +This option allows you to run RAG on a GPU using an open-source LLM from HuggingFace. Here's a quick [tutorial](#ec2-dev-setup) on setting up an EC2 GPU instance for the installation below. -If you are on EC2, you can launch a GPU instance with the following config: -- EC2 `g4dn.xlarge` (if you want to run a pretrained LLM with 7B parameters) -- Deep Learning AMI PyTorch GPU 2.0.1 (Ubuntu 20.04) - -- EBS: at least 100G - +On your GPU instance terminal, create a conda environment using: +``` +conda create -n pykoi python=3.10 -y && source activate pykoi +``` -Next, on your GPU instance terminal, create a conda environment using: +Then install `pykoi` and [pytorch based on your cuda version](https://pytorch.org/get-started/locally/). You can find your CUDA version via `nvcc -V`. +``` +pip3 install "pykoi[rag, huggingface]" + +pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121 # cu121 means cuda 12.1 +``` + +### Option 3: RLHF (GPU) +This option allows you to train LLM via RLHF on a GPU. Here's a quick [tutorial](#ec2-dev-setup) on setting up an EC2 GPU instance for the installation below. + +On your GPU instance terminal, create a conda environment using: ``` conda create -n pykoi python=3.10 -y && source activate pykoi ``` -Then install `pykoi` and [pytorch based on your cuda version](https://pytorch.org/get-started). +Then install `pykoi` and [pytorch based on your cuda version](https://pytorch.org/get-started/locally/). You can find your CUDA version via `nvcc -V`. ``` -pip3 install pykoi +pip3 install "pykoi[rlhf]" -# install torch based on cuda (e.g. cu118 means cuda 11.8) -pip3 install torch --index-url https://download.pytorch.org/whl/cu118 +pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121 # cu121 means cuda 12.1 ``` Congrats you have finished the installation! @@ -91,13 +100,41 @@ conda create -n pykoi python=3.10 conda activate pykoi cd pykoi pip3 install poetry -poetry install --no-root ``` +Then, based the feature you need to develop, run one or more installation options below. We recommend install all the options below although it may take ~3 minutes longer. + +- Option 1: RAG (CPU) + ``` + poetry install --no-root --extras rag + ``` +- Option 2: RAG (GPU) + ``` + poetry install --no-root --extras "rag huggingface" + ``` +- Option 3: RLHF (GPU) + ``` + poetry install --no-root --extras rlhf + ``` + +Finally, if you are on a GPU, install [pytorch based on your cuda version](https://pytorch.org/get-started/locally/). You can find your CUDA version via `nvcc -V`. +``` +pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121 # cu121 means cuda 12.1 +``` + + ### Frontend Dev Setup -Frontend: ``` cd pykoi/pykoi/frontend npm install npm run build ``` + +### EC2 Dev Setup +If you are on EC2, you can launch a GPU instance with the following config: +- EC2 `g4dn.xlarge` (if you want to run a pretrained LLM with 7B parameters) +- Deep Learning AMI PyTorch GPU 2.0.1 (Ubuntu 20.04) + +- EBS: at least 100G + + diff --git a/example/chatbot/demo_launch_app_cpu_openai.ipynb b/example/chatbot/demo_launch_app_cpu_openai.ipynb index 509f776..65c1060 100644 --- a/example/chatbot/demo_launch_app_cpu_openai.ipynb +++ b/example/chatbot/demo_launch_app_cpu_openai.ipynb @@ -1,5 +1,24 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Launch a Chatbot UI (with Database) from an OpenAI model\n", + "\n", + "`pykoi` provides simple UI to launch a chatbot UI based on your LLMs, including your own finetuned LLM, a pretrained LLM from huggingface, or OpenAI/Anthropic/Bedrock APIs. This demo shows how to create and launch an LLM chatbot UI and database for OpenAI/Anthropic/Bedrock APIs. Let's get started!\n", + "\n", + "\n", + "### Prerequisites\n", + "To run this jupyter notebook, you need a `pykoi` environment with the `rag` option. You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-1-rag-cpu) to set up the environment. \n", + "\n", + "You may also need `pip install ipykernel` to run the kernel environment.\n", + "\n", + "\n", + "### (Optional) Developer setup\n", + "If you are a normal user of `pykoi`, you can skip this step. However, if you modify the pykoi code and want to test your changes, you can uncomment the code below." + ] + }, { "cell_type": "code", "execution_count": null, @@ -9,19 +28,18 @@ "# %reload_ext autoreload\n", "# %autoreload 2\n", "\n", - "# import os\n", "# import sys\n", "\n", - "# # Add the root folder to the module search path\n", - "# # Get the current directory\n", - "# current_directory = os.getcwd()\n", - "\n", - "# # Move two levels up (go to the parent directory of the parent directory)\n", - "# two_levels_up_directory = os.path.dirname(os.path.dirname(current_directory))\n", - "\n", - "# print(two_levels_up_directory)\n", - "\n", - "# sys.path.append(two_levels_up_directory)" + "# sys.path.append(\".\")\n", + "# sys.path.append(\"..\")\n", + "# sys.path.append(\"../..\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import Libraries" ] }, { @@ -30,7 +48,6 @@ "metadata": {}, "outputs": [], "source": [ - "## pip install ipykernel\n", "from pykoi import Application\n", "from pykoi.chat import ModelFactory\n", "from pykoi.chat import QuestionAnswerDatabase\n", diff --git a/example/chatbot/openai_model_demo.py b/example/chatbot/demo_launch_app_cpu_openai.py similarity index 65% rename from example/chatbot/openai_model_demo.py rename to example/chatbot/demo_launch_app_cpu_openai.py index a9f6956..35d6646 100644 --- a/example/chatbot/openai_model_demo.py +++ b/example/chatbot/demo_launch_app_cpu_openai.py @@ -1,4 +1,18 @@ -"""Demo for the chatbot application using OpenAI endpoint.""" +""" +Demo for launching a chatbot UI (with database) from an OpenAI model. + +- Prerequisites: + To run this jupyter notebook, you need a `pykoi` environment with the `rag` option. + You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-1-rag-cpu) + to set up the environment. +- Run the demo: + 1. Enter your OpenAI API key in the `api_key` below. + 2. On terminal and `~/pykoi` directory, run + ``` + python -m example.chatbot.demo_launch_app_cpu_openai + ``` +""" + from pykoi import Application from pykoi.chat import ModelFactory from pykoi.chat import QuestionAnswerDatabase @@ -9,7 +23,7 @@ # Creating an OpenAI model (requires an OpenAI API key) # ########################################################## # enter openai api key here -api_key = "" +api_key = "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" # Creating an OpenAI model model = ModelFactory.create_model( diff --git a/example/chatbot/demo_launch_app_gpu_huggingface.ipynb b/example/chatbot/demo_launch_app_gpu_huggingface.ipynb index 35768ff..4b3246e 100644 --- a/example/chatbot/demo_launch_app_gpu_huggingface.ipynb +++ b/example/chatbot/demo_launch_app_gpu_huggingface.ipynb @@ -1,5 +1,24 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Launch a Chatbot UI (with Database) from Open-source LLMs\n", + "\n", + "`pykoi` provides simple UI to launch a chatbot UI based on your LLMs, including your own finetuned LLM, a pretrained LLM from huggingface, or OpenAI/Anthropic/Bedrock APIs. This demo shows how to create and launch an LLM chatbot UI and database (with Database) from Open-source LLMs from Huggingface. Let's get started!\n", + "\n", + "\n", + "### Prerequisites\n", + "To run this jupyter notebook, you need a `pykoi` environment with the `huggingface` option. You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-2-rag-gpu) to set up the environment. \n", + "\n", + "You may also need `pip install ipykernel` to run the kernel environment.\n", + "\n", + "\n", + "### (Optional) Developer setup\n", + "If you are a normal user of `pykoi`, you can skip this step. However, if you modify the pykoi code and want to test your changes, you can uncomment the code below." + ] + }, { "cell_type": "code", "execution_count": null, @@ -9,19 +28,18 @@ "# %reload_ext autoreload\n", "# %autoreload 2\n", "\n", - "# import os\n", "# import sys\n", "\n", - "# # Add the root folder to the module search path\n", - "# # Get the current directory\n", - "# current_directory = os.getcwd()\n", - "\n", - "# # Move two levels up (go to the parent directory of the parent directory)\n", - "# two_levels_up_directory = os.path.dirname(os.path.dirname(current_directory))\n", - "\n", - "# print(two_levels_up_directory)\n", - "\n", - "# sys.path.append(two_levels_up_directory)" + "# sys.path.append(\".\")\n", + "# sys.path.append(\"..\")\n", + "# sys.path.append(\"../..\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import Libraries" ] }, { diff --git a/example/chatbot/huggingface_model_demo.py b/example/chatbot/demo_launch_app_gpu_huggingface.py similarity index 71% rename from example/chatbot/huggingface_model_demo.py rename to example/chatbot/demo_launch_app_gpu_huggingface.py index e2cbc0e..fe72694 100644 --- a/example/chatbot/huggingface_model_demo.py +++ b/example/chatbot/demo_launch_app_gpu_huggingface.py @@ -1,4 +1,16 @@ -"""Demo for the chatbot application.""" +""" +Demo for the chatbot application using open source LLMs from Huggingface. + +- Prerequisites: + To run this jupyter notebook, you need a `pykoi` environment with the `huggingface` option. + You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-2-rag-gpu) + to set up the environment. +- Run the demo: + 1. On terminal and `~/pykoi` directory, run + ``` + python -m example.chatbot.demo_launch_app_gpu_huggingface + ``` +""" from pykoi import Application from pykoi.chat import ModelFactory from pykoi.chat import QuestionAnswerDatabase diff --git a/example/chatbot/peft_huggingface_model_demo.py b/example/chatbot/demo_launch_app_gpu_huggingface_peft.py similarity index 72% rename from example/chatbot/peft_huggingface_model_demo.py rename to example/chatbot/demo_launch_app_gpu_huggingface_peft.py index be4b8b5..fe82e47 100644 --- a/example/chatbot/peft_huggingface_model_demo.py +++ b/example/chatbot/demo_launch_app_gpu_huggingface_peft.py @@ -1,4 +1,17 @@ -"""Demo for the chatbot application.""" +""" +Demo for the chatbot application using open source LLMs from Huggingface Peft. + +- Prerequisites: + To run this jupyter notebook, you need a `pykoi` environment with the `huggingface` option. + You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-2-rag-gpu) + to set up the environment. +- Run the demo: + 1. On terminal and `~/pykoi` directory, run + ``` + python -m example.chatbot.demo_launch_app_gpu_huggingface_peft + ``` +""" + from pykoi import Application from pykoi.chat import ModelFactory from pykoi.chat import QuestionAnswerDatabase diff --git a/example/chatbot/demo_model_comparator_cpu_openai.ipynb b/example/comparator/demo_model_comparator_cpu_openai.ipynb similarity index 76% rename from example/chatbot/demo_model_comparator_cpu_openai.ipynb rename to example/comparator/demo_model_comparator_cpu_openai.ipynb index 19a84d1..e75aed6 100644 --- a/example/chatbot/demo_model_comparator_cpu_openai.ipynb +++ b/example/comparator/demo_model_comparator_cpu_openai.ipynb @@ -1,5 +1,23 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Demo: LLMs Comparison between OpenAI models\n", + "\n", + "`pykoi` provides simple API to compare between LLMs, including your own finetuned LLM, a pretrained LLM from huggingface, or OpenAI/Anthropic/Bedrock APIs. This demo shows how to create and launch an LLM comparison app for OpenAI/Anthropic/Bedrock APIs. Let's get started!\n", + "\n", + "### Prerequisites\n", + "To run this jupyter notebook, you need a `pykoi` environment with the `rag` option. You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-1-rag-cpu) to set up the environment. \n", + "\n", + "You may also need `pip install ipykernel` to run the kernel environment.\n", + "\n", + "\n", + "### (Optional) Developer setup\n", + "If you are a normal user of `pykoi`, you can skip this step. However, if you modify the pykoi code and want to test your changes, you can uncomment the code below." + ] + }, { "cell_type": "code", "execution_count": null, @@ -9,19 +27,18 @@ "# %reload_ext autoreload\n", "# %autoreload 2\n", "\n", - "# import os\n", "# import sys\n", "\n", - "# # Add the root folder to the module search path\n", - "# # Get the current directory\n", - "# current_directory = os.getcwd()\n", - "\n", - "# # Move two levels up (go to the parent directory of the parent directory)\n", - "# two_levels_up_directory = os.path.dirname(os.path.dirname(current_directory))\n", - "\n", - "# print(two_levels_up_directory)\n", - "\n", - "# sys.path.append(two_levels_up_directory)" + "# sys.path.append(\".\")\n", + "# sys.path.append(\"..\")\n", + "# sys.path.append(\"../..\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import Libraries" ] }, { @@ -39,13 +56,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Demo: LLMs Comparison\n", - "\n", - "`pykoi` provides simple API to compare between LLMs, including your own finetuned LLM, a pretrained LLM from huggingface, or OpenAI/Anthropic/Bedrock APIs.\n", "\n", - "This demo shows how to create and launch an LLM comparison app. Let's get started!\n", "\n", - "## Load LLMs\n", + "### Load LLMs\n", "\n", "#### 1. Creating an OpenAI model (requires an OpenAI API key)" ] @@ -108,6 +121,13 @@ "nest_asyncio.apply()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Launch the App" + ] + }, { "cell_type": "code", "execution_count": null, @@ -137,13 +157,6 @@ " \n", "" ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { diff --git a/example/chatbot/demo_model_comparator_cpu_openai.py b/example/comparator/demo_model_comparator_cpu_openai.py similarity index 80% rename from example/chatbot/demo_model_comparator_cpu_openai.py rename to example/comparator/demo_model_comparator_cpu_openai.py index b5c45f9..483c3f3 100644 --- a/example/chatbot/demo_model_comparator_cpu_openai.py +++ b/example/comparator/demo_model_comparator_cpu_openai.py @@ -1,4 +1,18 @@ -"""Demo for the chatbot application using multiple model endpoint.""" +""" +Demo for the chatbot application using multiple OpenAI models. + +- Prerequisites: + To run this jupyter notebook, you need a `pykoi` environment with the `rag` option. + You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-1-rag-cpu) + to set up the environment. +- Run the demo: + 1. Enter your OpenAI API key in the `api_key` below. + 2. On terminal and `~/pykoi` directory, run + ``` + python -m example.comparator.demo_model_comparator_cpu_openai + ``` +""" + from pykoi import Application from pykoi.chat import ModelFactory from pykoi.component import Compare diff --git a/example/chatbot/demo_model_comparator_gpu_huggingface.ipynb b/example/comparator/demo_model_comparator_gpu_huggingface.ipynb similarity index 76% rename from example/chatbot/demo_model_comparator_gpu_huggingface.ipynb rename to example/comparator/demo_model_comparator_gpu_huggingface.ipynb index 92614e9..1799762 100644 --- a/example/chatbot/demo_model_comparator_gpu_huggingface.ipynb +++ b/example/comparator/demo_model_comparator_gpu_huggingface.ipynb @@ -1,5 +1,23 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Demo: LLMs Comparison between Open-source LLMs\n", + "\n", + "`pykoi` provides simple API to compare between LLMs, including your own finetuned LLM, a pretrained LLM from huggingface, or OpenAI/Anthropic/Bedrock APIs. This demo shows how to create and launch an LLM comparison app between Open-source LLMs from huggingface. Let's get started!\n", + "\n", + "### Prerequisites\n", + "To run this jupyter notebook, you need a `pykoi` environment with the `huggingface` option. You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-2-rag-gpu) to set up the environment. \n", + "\n", + "You may also need `pip install ipykernel` to run the kernel environment.\n", + "\n", + "\n", + "### (Optional) Developer setup\n", + "If you are a normal user of `pykoi`, you can skip this step. However, if you modify the pykoi code and want to test your changes, you can uncomment the code below." + ] + }, { "cell_type": "code", "execution_count": null, @@ -9,19 +27,18 @@ "# %reload_ext autoreload\n", "# %autoreload 2\n", "\n", - "# import os\n", "# import sys\n", "\n", - "# # Add the root folder to the module search path\n", - "# # Get the current directory\n", - "# current_directory = os.getcwd()\n", - "\n", - "# # Move two levels up (go to the parent directory of the parent directory)\n", - "# two_levels_up_directory = os.path.dirname(os.path.dirname(current_directory))\n", - "\n", - "# print(two_levels_up_directory)\n", - "\n", - "# sys.path.append(two_levels_up_directory)" + "# sys.path.append(\".\")\n", + "# sys.path.append(\"..\")\n", + "# sys.path.append(\"../..\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import Libraries" ] }, { @@ -42,21 +59,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Demo: LLMs Comparison\n", - "\n", - "`pykoi` provides simple API to compare between LLMs, including your own finetuned LLM, a pretrained LLM from huggingface, or OpenAI/Anthropic/Bedrock APIs.\n", - "\n", - "This demo shows how to create and launch an LLM comparison app. Let's get started!\n", - "\n", - "## Load LLMs\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Creating a Huggingface model (requires at least EC2 `g4.2xlarge` or GPU with 16G memory)" + "### Load LLMs\n", + "#### Creating a Huggingface model (requires at least EC2 `g4dn.xlarge` or GPU with at least 16G memory)" ] }, { @@ -65,7 +69,7 @@ "metadata": {}, "outputs": [], "source": [ - "## requires a GPU with at least 16GB memory (e.g. g4.2xlarge)\n", + "## requires a GPU with at least 16GB memory (e.g. g4dn.xlarge)\n", "huggingface_model_1 = ModelFactory.create_model(\n", " model_source=\"huggingface\",\n", " pretrained_model_name_or_path=\"tiiuae/falcon-rw-1b\",\n", @@ -78,7 +82,7 @@ "metadata": {}, "outputs": [], "source": [ - "## requires a GPU with at least 16GB memory (e.g. g4.2xlarge)\n", + "## requires a GPU with at least 16GB memory (e.g. g4dn.2xlarge)\n", "huggingface_model_2 = ModelFactory.create_model(\n", " model_source=\"huggingface\",\n", " pretrained_model_name_or_path=\"databricks/dolly-v2-3b\",\n", @@ -102,9 +106,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Create a chatbot comparator\n", + "### Create a chatbot comparator\n", "\n", - "### Add `nest_asyncio` \n", + "#### Add `nest_asyncio` \n", "Add `nest_asyncio` to avoid error. Since we're running another interface inside a Jupyter notebook where an asyncio event loop is already running, we'll encounter the error. (since The uvicorn.run() function uses asyncio.run(), which isn't compatible with a running event loop.)" ] }, @@ -134,7 +138,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Add ngrok auth (TODO: change to bash file)" + "#### Add ngrok auth (TODO: change to bash file)" ] }, { diff --git a/example/chatbot/demo_model_comparator_gpu_huggingface.py b/example/comparator/demo_model_comparator_gpu_huggingface.py similarity index 81% rename from example/chatbot/demo_model_comparator_gpu_huggingface.py rename to example/comparator/demo_model_comparator_gpu_huggingface.py index d033143..7ddec16 100644 --- a/example/chatbot/demo_model_comparator_gpu_huggingface.py +++ b/example/comparator/demo_model_comparator_gpu_huggingface.py @@ -1,4 +1,16 @@ -"""Demo for the chatbot application using multiple model endpoint.""" +""" +Demo for the chatbot application using multiple open source LLMs from Huggingface. + +- Prerequisites: + To run this jupyter notebook, you need a `pykoi` environment with the `huggingface` option. + You can follow [the installation guide](https://github.com/CambioML/pykoi/tree/install#option-2-rag-gpu) + to set up the environment. +- Run the demo: + 1. On terminal and `~/pykoi` directory, run + ``` + python -m example.comparator.demo_model_comparator_gpu_huggingface + ``` +""" from pykoi import Application from pykoi.chat import ModelFactory from pykoi.component import Compare diff --git a/example/flex/flex_demo.py b/example/flex/flex_demo.py index 13c0d09..60bf66c 100644 --- a/example/flex/flex_demo.py +++ b/example/flex/flex_demo.py @@ -8,6 +8,7 @@ from pykoi.retrieval import RetrievalFactory from pykoi.retrieval import VectorDbFactory from pykoi.component import Chatbot, Dashboard, RetrievalQA +from pykoi.chat import RAGDatabase load_dotenv() @@ -32,9 +33,9 @@ def main(**kargs): ) # retrieval, chatbot, and dashboard pykoi components - retriever = RetrievalQA(retrieval_model=retrieval_model, vector_db=vector_db) - chatbot = Chatbot(None, feedback="vote", is_retrieval=True) - dashboard = Dashboard(QuestionAnswerDatabase()) + retriever = RetrievalQA(retrieval_model=retrieval_model, vector_db=vector_db, feedback="rag") + chatbot = Chatbot(None, feedback="rag", is_retrieval=True) + dashboard = Dashboard(RAGDatabase(), feedback="rag") ############################################################ # Starting the application and retrieval qa as a component # diff --git a/pykoi/__init__.py b/pykoi/__init__.py index 86c1474..827a46b 100644 --- a/pykoi/__init__.py +++ b/pykoi/__init__.py @@ -1,4 +1,3 @@ from pykoi.application import Application - -__version__ = "0.0.6" +__version__ = "0.0.7" diff --git a/pykoi/application.py b/pykoi/application.py index 50d2850..0bdbf1a 100644 --- a/pykoi/application.py +++ b/pykoi/application.py @@ -20,6 +20,7 @@ from pykoi.interactives.chatbot import Chatbot from pykoi.telemetry.telemetry import Telemetry from pykoi.telemetry.events import AppStartEvent, AppStopEvent +from pykoi.chat.db.constants import QA_LIST_SEPARATOR oauth_scheme = HTTPBasic() @@ -29,6 +30,17 @@ class UpdateQATable(BaseModel): id: int vote_status: str +class UpdateRAGTable(BaseModel): + id: int + vote_status: str + +class UpdateQATableAnswer(BaseModel): + id: int + new_answer: str + +class UpdateRAGTableAnswer(BaseModel): + id: int + new_answer: str class RankingTableUpdate(BaseModel): question: str @@ -212,6 +224,7 @@ async def update_qa_table( user: Union[None, UserInDB] = Depends(self.get_auth_dependency()), ): try: + print("updating QA vote") component["component"].database.update_vote_status( request_body.id, request_body.vote_status ) @@ -219,6 +232,20 @@ async def update_qa_table( except Exception as ex: return {"log": f"Table update failed: {ex}", "status": "500"} + @app.post("/chat/qa_table/update_answer") + async def update_qa_table_response( + request_body: UpdateQATableAnswer, + user: Union[None, UserInDB] = Depends(self.get_auth_dependency()), + ): + try: + component["component"].database.update_answer( + request_body.id, request_body.new_answer + ) + print("/chat/qa_table/update_answer", request_body.id, request_body.new_answer) + return {"log": "Table response updated", "new_answer": request_body.new_answer, "status": "200"} + except Exception as ex: + return {"log": f"Table update failed: {ex}", "status": "500"} + @app.get("/chat/qa_table/close") async def close_qa_table( user: Union[None, UserInDB] = Depends(self.get_auth_dependency()) @@ -274,6 +301,51 @@ async def retrieve_ranking_table( except Exception as ex: return {"log": f"Table retrieval failed: {ex}", "status": "500"} + @app.post("/chat/rag_table/update") + async def update_rag_table( + request_body: UpdateRAGTable, + user: Union[None, UserInDB] = Depends(self.get_auth_dependency()), + ): + try: + print("updating RAG vote") + component["component"].database.update_vote_status( + request_body.id, request_body.vote_status + ) + return {"log": "Table updated", "status": "200"} + except Exception as ex: + return {"log": f"Table update failed: {ex}", "status": "500"} + + @app.post("/chat/rag_table/update_answer") + async def update_rag_table_response( + request_body: UpdateRAGTableAnswer, + user: Union[None, UserInDB] = Depends(self.get_auth_dependency()), + ): + try: + component["component"].database.update_answer( + request_body.id, request_body.new_answer + ) + print("/chat/rag_table/update_answer", request_body.id, request_body.new_answer) + return {"log": "Table response updated", "new_answer": request_body.new_answer, "status": "200"} + except Exception as ex: + return {"log": f"Table update failed: {ex}", "status": "500"} + + @app.get("/chat/rag_table/retrieve") + async def retrieve_rag_table( + user: Union[None, UserInDB] = Depends(self.get_auth_dependency()) + ): + try: + rows = component["component"].database.retrieve_all_question_answers() + modified_rows = [] + for row in rows: + row_list = list(row) # Convert the tuple to a list + row_list[5] = row_list[5].split(QA_LIST_SEPARATOR) + row_list[6] = row_list[6].split(QA_LIST_SEPARATOR) + row_list[7] = row_list[7].split(QA_LIST_SEPARATOR) + modified_rows.append(row_list) # Append the modified list to the new list + return {"rows": modified_rows, "log": "RAG Table retrieved", "status": "200"} + except Exception as ex: + return {"log": f"Table retrieval failed: {ex}", "status": "500"} + def create_feedback_route(self, app: FastAPI, component: Dict[str, Any]): """ Create feedback routes for the application. @@ -491,28 +563,20 @@ async def inference( try: print("[/retrieval]: model inference.....", request_body.prompt) component["component"].retrieval_model.re_init(request_body.file_names) - output = component[ - "component" - ].retrieval_model.run_with_return_source_documents( - {"query": request_body.prompt} - ) - id = component["component"].database.insert_question_answer( - request_body.prompt, output["result"] - ) - print("output", output, output["result"]) + output = component["component"].retrieval_model.run_with_return_source_documents({"query": request_body.prompt}) + print('output', output, output["result"]) if output["source_documents"] == []: - source = "N/A" - source_content = "N/A" + source = ["N/A"] + source_content = ["N/A"] else: - source = output["source_documents"][0].metadata.get( - "file_name", "No file name found" - ) - source_content = ( - "1. " - + output["source_documents"][0].page_content - + "\n2. " - + output["source_documents"][1].page_content - ) + source = [] + source_content = [] + for source_document in output["source_documents"]: + source.append(source_document.metadata.get('file_name', 'No file name found')) + source_content.append(source_document.page_content) + id = component["component"].database.insert_question_answer( + request_body.prompt, output["result"], request_body.file_names, source, source_content + ) return { "id": id, "log": "Inference complete", diff --git a/pykoi/chat/__init__.py b/pykoi/chat/__init__.py index cfdb23d..efbeb29 100644 --- a/pykoi/chat/__init__.py +++ b/pykoi/chat/__init__.py @@ -3,3 +3,4 @@ from pykoi.chat.llm.model_factory import ModelFactory from pykoi.chat.db.qa_database import QuestionAnswerDatabase from pykoi.chat.db.ranking_database import RankingDatabase +from pykoi.chat.db.rag_database import RAGDatabase \ No newline at end of file diff --git a/pykoi/chat/db/constants.py b/pykoi/chat/db/constants.py index 2c4e94d..432109e 100644 --- a/pykoi/chat/db/constants.py +++ b/pykoi/chat/db/constants.py @@ -25,3 +25,6 @@ RANKING_CSV_HEADER_UP_RANKING_ANSWER, RANKING_CSV_HEADER_LOW_RANKING_ANSWER, ) + +# list separator +QA_LIST_SEPARATOR = "||" \ No newline at end of file diff --git a/pykoi/chat/db/rag_database.py b/pykoi/chat/db/rag_database.py new file mode 100644 index 0000000..52a82c7 --- /dev/null +++ b/pykoi/chat/db/rag_database.py @@ -0,0 +1,244 @@ +"""Question answer database module""" +import csv +import datetime +import os +import sqlite3 +import threading + +import pandas as pd + +from pykoi.chat.db.constants import QA_CSV_HEADER, QA_LIST_SEPARATOR + + +class RAGDatabase: + """RAG Question Answer Database class""" + + def __init__( + self, + db_file: str = os.path.join(os.getcwd(), "rag.db"), + debug: bool = False, + ): + """ + Initializes a new instance of the RAGDatabase class. + + Args: + db_file (str): The path to the SQLite database file. + debug (bool, optional): Whether to print debug messages. Defaults to False. + """ + self._db_file = db_file + self._debug = debug + self._local = threading.local() # Thread-local storage + self._lock = threading.Lock() # Lock for concurrent write operations + self.create_table() + + def get_connection(self): + """Returns the thread-local database connection""" + if not hasattr(self._local, "connection"): + self._local.connection = sqlite3.connect(self._db_file) + return self._local.connection + + def get_cursor(self): + """Returns the thread-local database cursor""" + if not hasattr(self._local, "cursor"): + self._local.cursor = self.get_connection().cursor() + return self._local.cursor + + def create_table(self): + """ + Creates the question_answer table if it does not already exist in the database. + The table has 6 columns: id (primary key), question, answer, source, source_content, and vote_status. + vote_status is a text field that can only have the values 'up', 'down', or 'n/a'. + """ + query = """ + CREATE TABLE IF NOT EXISTS rag_question_answer ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + question TEXT, + answer TEXT, + edited_answer TEXT, + vote_status TEXT CHECK (vote_status IN ('up', 'down', 'n/a')), + rag_sources TEXT, + source TEXT, + source_content TEXT, + timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP + ); + """ + with self._lock: + cursor = self.get_cursor() + cursor.execute(query) + self.get_connection().commit() + + if self._debug: + rows = self.retrieve_all_question_answers() + print("Table contents after creating table:") + self.print_table(rows) + + def insert_question_answer(self, question: str, answer: str, rag_sources: list, source: list, source_content: list): + """ + Inserts a new question-answer pair into the database with the given question and answer. + The vote_status field is set to 'n/a' by default. + Returns the ID of the newly inserted row. + + Args: + question (str): The question to insert. + answer (str): The answer to insert. + rag_sources (list): The RAG sources of the question-answer pair. + source (list): The source(s) of the question-answer pair. + source_content (list): The source content(s) of the question-answer pair. + + Returns: + int: The ID of the newly inserted row. + """ + timestamp = datetime.datetime.now() + query = """ + INSERT INTO rag_question_answer (question, answer, edited_answer, rag_sources, source, source_content, vote_status, timestamp) + VALUES (?, ?, '', ?, ?, ?, 'n/a', ?); + """ + rag_sources = QA_LIST_SEPARATOR.join(rag_sources) + source = QA_LIST_SEPARATOR.join(source) + source_content = QA_LIST_SEPARATOR.join(source_content) + print("rag insert question answer", rag_sources) + + with self._lock: + cursor = self.get_cursor() + cursor.execute(query, (question, answer, rag_sources, source, source_content, timestamp)) + self.get_connection().commit() + + if self._debug: + rows = self.retrieve_all_question_answers() + print("Table contents after inserting table:") + self.print_table(rows) + + return cursor.lastrowid + + def update_vote_status(self, id, vote_status): + """ + Updates the vote status of a question-answer pair with the given ID. + + Args: + id (int): The ID of the question-answer pair to update. + vote_status (str): The new vote status to set. Must be one of 'up', 'down', or 'n/a'. + + Raises: + ValueError: If the question with the given ID does not exist. + """ + query = """ + UPDATE rag_question_answer + SET vote_status = ? + WHERE id = ?; + """ + with self._lock: + cursor = self.get_cursor() + cursor.execute(query, (vote_status, id)) + self.get_connection().commit() + + if cursor.rowcount == 0: + raise ValueError(f"Question with ID {id} does not exist.") + + if self._debug: + rows = self.retrieve_all_question_answers() + print("Table contents after updating table:") + self.print_table(rows) + + def update_answer(self, id, new_answer): + """ + Updates the edited_answer of a question-answer pair with the given ID. + + Args: + id (int): The ID of the question-answer pair to update. + new_answer (str): The new modified response to set. + + Raises: + ValueError: If the question with the given ID does not exist. + """ + query = """ + UPDATE rag_question_answer + SET edited_answer = ? + WHERE id = ?; + """ + print('update_answer',new_answer) + with self._lock: + cursor = self.get_cursor() + cursor.execute(query, (new_answer, id)) + self.get_connection().commit() + + if cursor.rowcount == 0: + raise ValueError(f"Question with ID {id} does not exist.") + + if self._debug: + rows = self.retrieve_all_question_answers() + print("Table contents after updating table:") + self.print_table(rows) + + def retrieve_all_question_answers(self): + """ + Retrieves all question-answer pairs from the database. + + Returns: + list: A list of tuples representing the question-answer pairs. + """ + query = """ + SELECT * FROM rag_question_answer; + """ + with self._lock: + cursor = self.get_cursor() + cursor.execute(query) + rows = cursor.fetchall() + return rows + + def retrieve_all_question_answers_as_pandas(self): + """ + Retrieves all question-answer pairs from the database as a pandas dataframe. + + Returns: + DataFrame: A pandas dataframe. + """ + rows = self.retrieve_all_question_answers() + rows_to_pd = pd.DataFrame(rows) + rows_to_pd.columns = QA_CSV_HEADER + return rows_to_pd + + def close_connection(self): + """ + Closes the connection to the database. + """ + if hasattr(self._local, "cursor"): + self._local.cursor.close() + del self._local.cursor + if hasattr(self._local, "connection"): + self._local.connection.close() + del self._local.connection + + def print_table(self, rows): + """ + Prints the contents of the table in a formatted manner. + + Args: + rows (list): A list of tuples where each tuple represents a row in the table. + Each tuple contains five elements: ID, Question, Answer, Timestamp, and Vote Status. + """ + for row in rows: + print( + f"ID: {row[0]}, Question: {row[1]}, " + f"Answer: {row[2]}, Vote Status: {row[3]}, Timestamp: {row[4]}" + ) + + def save_to_csv(self, csv_file_name="question_answer_votes.csv"): + """ + This method saves the contents of the question_answer table into a CSV file. + + Args: + csv_file_name (str, optional): The name of the CSV file to which the data will be written. + Defaults to "question_answer_votes.csv". + + The CSV file will have the following columns: ID, Question, Answer, Vote Status. Each row in the + CSV file corresponds to a row in the question_answer table. + + This method first retrieves all question-answer pairs from the database by calling the + retrieve_all_question_answers method. It then writes this data to the CSV file. + """ + my_sql_data = self.retrieve_all_question_answers() + + with open(csv_file_name, "w", newline="") as file: + writer = csv.writer(file) + writer.writerow(QA_CSV_HEADER) + writer.writerows(my_sql_data) diff --git a/pykoi/component/base.py b/pykoi/component/base.py index 6f6e15e..44960f3 100644 --- a/pykoi/component/base.py +++ b/pykoi/component/base.py @@ -5,6 +5,7 @@ from pykoi.component.chatbot_database_factory import ChatbotDatabaseFactory from pykoi.component.constants import FeedbackType from pykoi.chat.db.qa_database import QuestionAnswerDatabase +from pykoi.chat.db.rag_database import RAGDatabase from pykoi.chat.db.ranking_database import RankingDatabase from pykoi.chat.llm.abs_llm import AbsLlm @@ -129,7 +130,7 @@ class Dashboard(Component): database (str): The database to use for the dashboard. """ - def __init__(self, database: QuestionAnswerDatabase, **kwargs): + def __init__(self, database: any, **kwargs): """ Initialize a new instance of Dashboard. diff --git a/pykoi/component/chatbot_database_factory.py b/pykoi/component/chatbot_database_factory.py index 0c158e3..d774ba8 100644 --- a/pykoi/component/chatbot_database_factory.py +++ b/pykoi/component/chatbot_database_factory.py @@ -4,6 +4,7 @@ from pykoi.component.constants import FeedbackType from pykoi.chat.db.qa_database import QuestionAnswerDatabase from pykoi.chat.db.ranking_database import RankingDatabase +from pykoi.chat.db.rag_database import RAGDatabase class ChatbotDatabaseFactory: @@ -20,13 +21,15 @@ def create(feedback: Union[str, FeedbackType]): feedback (Union[str, FeedbackType]): The type of the chatbot. Returns: - Union[QuestionAnswerDatabase, RankingDatabase]: The created database. + Union[QuestionAnswerDatabase, RankingDatabase, RAGDatabase]: The created database. """ feedback = FeedbackType(feedback) if feedback == FeedbackType.VOTE: return QuestionAnswerDatabase() elif feedback == FeedbackType.RANK: return RankingDatabase() + elif feedback == FeedbackType.RAG: + return RAGDatabase() else: raise ValueError( f"Invalid feedback name: {feedback}. " diff --git a/pykoi/component/constants.py b/pykoi/component/constants.py index 5d78872..f1f30ee 100644 --- a/pykoi/component/constants.py +++ b/pykoi/component/constants.py @@ -7,3 +7,4 @@ class FeedbackType(Enum): VOTE = "vote" RANK = "rank" + RAG = "rag" diff --git a/pykoi/frontend/dist/assets/index-47e316ee.js b/pykoi/frontend/dist/assets/index-47e316ee.js deleted file mode 100644 index 84a03c6..0000000 --- a/pykoi/frontend/dist/assets/index-47e316ee.js +++ /dev/null @@ -1,65 +0,0 @@ -(function(){const t=document.createElement("link").relList;if(t&&t.supports&&t.supports("modulepreload"))return;for(const i of document.querySelectorAll('link[rel="modulepreload"]'))r(i);new MutationObserver(i=>{for(const o of i)if(o.type==="childList")for(const l of o.addedNodes)l.tagName==="LINK"&&l.rel==="modulepreload"&&r(l)}).observe(document,{childList:!0,subtree:!0});function n(i){const o={};return i.integrity&&(o.integrity=i.integrity),i.referrerPolicy&&(o.referrerPolicy=i.referrerPolicy),i.crossOrigin==="use-credentials"?o.credentials="include":i.crossOrigin==="anonymous"?o.credentials="omit":o.credentials="same-origin",o}function r(i){if(i.ep)return;i.ep=!0;const o=n(i);fetch(i.href,o)}})();function X(){}const Hl=e=>e;function Lr(e,t){for(const n in t)e[n]=t[n];return e}function Ll(e){return e()}function qr(){return Object.create(null)}function we(e){e.forEach(Ll)}function ot(e){return typeof e=="function"}function de(e,t){return e!=e?t==t:e!==t||e&&typeof e=="object"||typeof e=="function"}function Zs(e){return Object.keys(e).length===0}function fr(e,...t){if(e==null)return X;const n=e.subscribe(...t);return n.unsubscribe?()=>n.unsubscribe():n}function ea(e){let t;return fr(e,n=>t=n)(),t}function ie(e,t,n){e.$$.on_destroy.push(fr(t,n))}function Ue(e,t,n){return e.set(n),t}function qn(e){return e&&ot(e.destroy)?e.destroy:X}const ql=typeof window<"u";let Gl=ql?()=>window.performance.now():()=>Date.now(),gi=ql?e=>requestAnimationFrame(e):X;const Ot=new Set;function Bl(e){Ot.forEach(t=>{t.c(e)||(Ot.delete(t),t.f())}),Ot.size!==0&&gi(Bl)}function jl(e){let t;return Ot.size===0&&gi(Bl),{promise:new Promise(n=>{Ot.add(t={c:e,f:n})}),abort(){Ot.delete(t)}}}let cr=!1;function ta(){cr=!0}function na(){cr=!1}function ra(e,t,n,r){for(;e{u|=1<
e;const n=typeof e;if(n!==typeof t||Array.isArray(e)!==Array.isArray(t))throw new Error("Cannot interpolate values of different type");if(Array.isArray(e)){const r=t.map((i,o)=>jr(e[o],i));return i=>r.map(o=>o(i))}if(n==="object"){if(!e||!t)throw new Error("Object cannot be null");if(Pi(e)&&Pi(t)){e=e.getTime(),t=t.getTime();const o=t-e;return l=>new Date(e+l*o)}const r=Object.keys(t),i={};return r.forEach(o=>{i[o]=jr(e[o],t[o])}),o=>{const l={};return r.forEach(s=>{l[s]=i[s](o)}),l}}if(n==="number"){const r=t-e;return i=>e+i*r}throw new Error(`Cannot interpolate ${n} values`)}function mi(e,t={}){const n=Me(e);let r,i=e;function o(l,s){if(e==null)return n.set(e=l),Promise.resolve();i=l;let a=r,u=!1,{delay:f=0,duration:c=400,easing:g=Hl,interpolate:d=jr}=Lr(Lr({},t),s);if(c===0)return a&&(a.abort(),a=null),n.set(e=i),Promise.resolve();const p=Gl()+f;let m;return r=jl(_=>{if(_
c?(n.set(e=l),!1):(n.set(e=m(g(b/c))),!0)}),r.promise}return{set:o,update:(l,s)=>o(l(i,e),s),subscribe:n.subscribe}}const qe=Me([]),br=Me([]),yr=Me([]),et=Me("all"),Ni=Me({"n/a":1,up:1,down:1}),Na=["who","what","how","why","where","does","can","n/a"],Ur=mi(Na.map(e=>({question:e,count:0}))),Wr=Me(new Set);var Xr="http://www.w3.org/1999/xhtml";const Oi={svg:"http://www.w3.org/2000/svg",xhtml:Xr,xlink:"http://www.w3.org/1999/xlink",xml:"http://www.w3.org/XML/1998/namespace",xmlns:"http://www.w3.org/2000/xmlns/"};function Kl(e){var t=e+="",n=t.indexOf(":");return n>=0&&(t=e.slice(0,n))!=="xmlns"&&(e=e.slice(n+1)),Oi.hasOwnProperty(t)?{space:Oi[t],local:e}:e}function Oa(e){return function(){var t=this.ownerDocument,n=this.namespaceURI;return n===Xr&&t.documentElement.namespaceURI===Xr?t.createElement(e):t.createElementNS(n,e)}}function Ta(e){return function(){return this.ownerDocument.createElementNS(e.space,e.local)}}function Jl(e){var t=Kl(e);return(t.local?Ta:Oa)(t)}function Va(){}function Zl(e){return e==null?Va:function(){return this.querySelector(e)}}function za(e){typeof e!="function"&&(e=Zl(e));for(var t=this._groups,n=t.length,r=new Array(n),i=0;i =0;)(l=r[i])&&(o&&l.compareDocumentPosition(o)^4&&o.parentNode.insertBefore(l,o),o=l);return this}function cu(e){e||(e=du);function t(c,g){return c&&g?e(c.__data__,g.__data__):!c-!g}for(var n=this._groups,r=n.length,i=new Array(r),o=0;o Ask a question to receive an answer from the chatbot. If the response is
- satisfactory, click on the Ask a question and click on the better of the two responses. The better
- response will be outlined in green, the worse
- response outlined in red. This data will be
- automatically fed to RLHF. 0?d[o-1]:c,v.x1=o 0)for(o=0;o Ask a question and rank the answers across the models. Drag each answer to
- rank it, in ascending order, from left-to-right. Optionally, select the
- rank for each via the corresponding dropdown. {let d=g;for(const m of a.split(".")){var p;d=(p=d)==null?void 0:p[m]}return d}:f=g=>g[s.accessorKey]),!u)throw new Error;let c={id:`${String(u)}`,accessorFn:f,parent:r,depth:n,columnDef:s,columns:[],getFlatColumns:B(()=>[!0],()=>{var g;return[c,...(g=c.columns)==null?void 0:g.flatMap(d=>d.getFlatColumns())]},{key:"column.getFlatColumns",debug:()=>{var g;return(g=e.options.debugAll)!=null?g:e.options.debugColumns}}),getLeafColumns:B(()=>[e._getOrderColumnsFn()],g=>{var d;if((d=c.columns)!=null&&d.length){let p=c.columns.flatMap(m=>m.getLeafColumns());return g(p)}return[c]},{key:"column.getLeafColumns",debug:()=>{var g;return(g=e.options.debugAll)!=null?g:e.options.debugColumns}})};return c=e._features.reduce((g,d)=>Object.assign(g,d.createColumn==null?void 0:d.createColumn(c,e)),c),c}function fl(e,t,n){var r;let o={id:(r=n.id)!=null?r:t.id,column:t,index:n.index,isPlaceholder:!!n.isPlaceholder,placeholderId:n.placeholderId,depth:n.depth,subHeaders:[],colSpan:0,rowSpan:0,headerGroup:null,getLeafHeaders:()=>{const l=[],s=a=>{a.subHeaders&&a.subHeaders.length&&a.subHeaders.map(s),l.push(a)};return s(o),l},getContext:()=>({table:e,header:o,column:t})};return e._features.forEach(l=>{Object.assign(o,l.createHeader==null?void 0:l.createHeader(o,e))}),o}const Yg={createTable:e=>({getHeaderGroups:B(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.left,e.getState().columnPinning.right],(t,n,r,i)=>{var o,l;const s=(o=r==null?void 0:r.map(c=>n.find(g=>g.id===c)).filter(Boolean))!=null?o:[],a=(l=i==null?void 0:i.map(c=>n.find(g=>g.id===c)).filter(Boolean))!=null?l:[],u=n.filter(c=>!(r!=null&&r.includes(c.id))&&!(i!=null&&i.includes(c.id)));return Dn(t,[...s,...u,...a],e)},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterHeaderGroups:B(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.left,e.getState().columnPinning.right],(t,n,r,i)=>(n=n.filter(o=>!(r!=null&&r.includes(o.id))&&!(i!=null&&i.includes(o.id))),Dn(t,n,e,"center")),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftHeaderGroups:B(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.left],(t,n,r)=>{var i;const o=(i=r==null?void 0:r.map(l=>n.find(s=>s.id===l)).filter(Boolean))!=null?i:[];return Dn(t,o,e,"left")},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightHeaderGroups:B(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.right],(t,n,r)=>{var i;const o=(i=r==null?void 0:r.map(l=>n.find(s=>s.id===l)).filter(Boolean))!=null?i:[];return Dn(t,o,e,"right")},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getFooterGroups:B(()=>[e.getHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftFooterGroups:B(()=>[e.getLeftHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterFooterGroups:B(()=>[e.getCenterHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightFooterGroups:B(()=>[e.getRightHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getFlatHeaders:B(()=>[e.getHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftFlatHeaders:B(()=>[e.getLeftHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterFlatHeaders:B(()=>[e.getCenterHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightFlatHeaders:B(()=>[e.getRightHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterLeafHeaders:B(()=>[e.getCenterFlatHeaders()],t=>t.filter(n=>{var r;return!((r=n.subHeaders)!=null&&r.length)}),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftLeafHeaders:B(()=>[e.getLeftFlatHeaders()],t=>t.filter(n=>{var r;return!((r=n.subHeaders)!=null&&r.length)}),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightLeafHeaders:B(()=>[e.getRightFlatHeaders()],t=>t.filter(n=>{var r;return!((r=n.subHeaders)!=null&&r.length)}),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeafHeaders:B(()=>[e.getLeftHeaderGroups(),e.getCenterHeaderGroups(),e.getRightHeaderGroups()],(t,n,r)=>{var i,o,l,s,a,u;return[...(i=(o=t[0])==null?void 0:o.headers)!=null?i:[],...(l=(s=n[0])==null?void 0:s.headers)!=null?l:[],...(a=(u=r[0])==null?void 0:u.headers)!=null?a:[]].map(f=>f.getLeafHeaders()).flat()},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}})})};function Dn(e,t,n,r){var i,o;let l=0;const s=function(g,d){d===void 0&&(d=1),l=Math.max(l,d),g.filter(p=>p.getIsVisible()).forEach(p=>{var m;(m=p.columns)!=null&&m.length&&s(p.columns,d+1)},0)};s(e);let a=[];const u=(g,d)=>{const p={depth:d,id:[r,`${d}`].filter(Boolean).join("_"),headers:[]},m=[];g.forEach(_=>{const b=[...m].reverse()[0],v=_.column.depth===p.depth;let $,A=!1;if(v&&_.column.parent?$=_.column.parent:($=_.column,A=!0),b&&(b==null?void 0:b.column)===$)b.subHeaders.push(_);else{const k=fl(n,$,{id:[r,d,$.id,_==null?void 0:_.id].filter(Boolean).join("_"),isPlaceholder:A,placeholderId:A?`${m.filter(E=>E.column===$).length}`:void 0,depth:d,index:m.length});k.subHeaders.push(_),m.push(k)}p.headers.push(_),_.headerGroup=p}),a.push(p),d>0&&u(m,d-1)},f=t.map((g,d)=>fl(n,g,{depth:l,index:d}));u(f,l-1),a.reverse();const c=g=>g.filter(p=>p.column.getIsVisible()).map(p=>{let m=0,_=0,b=[0];p.subHeaders&&p.subHeaders.length?(b=[],c(p.subHeaders).forEach($=>{let{colSpan:A,rowSpan:k}=$;m+=A,b.push(k)})):m=1;const v=Math.min(...b);return _=_+v,p.colSpan=m,p.rowSpan=_,{colSpan:m,rowSpan:_}});return c((i=(o=a[0])==null?void 0:o.headers)!=null?i:[]),a}const xn={size:150,minSize:20,maxSize:Number.MAX_SAFE_INTEGER},Nr=()=>({startOffset:null,startSize:null,deltaOffset:null,deltaPercentage:null,isResizingColumn:!1,columnSizingStart:[]}),Qg={getDefaultColumnDef:()=>xn,getInitialState:e=>({columnSizing:{},columnSizingInfo:Nr(),...e}),getDefaultOptions:e=>({columnResizeMode:"onEnd",onColumnSizingChange:Ge("columnSizing",e),onColumnSizingInfoChange:Ge("columnSizingInfo",e)}),createColumn:(e,t)=>({getSize:()=>{var n,r,i;const o=t.getState().columnSizing[e.id];return Math.min(Math.max((n=e.columnDef.minSize)!=null?n:xn.minSize,(r=o??e.columnDef.size)!=null?r:xn.size),(i=e.columnDef.maxSize)!=null?i:xn.maxSize)},getStart:n=>{const r=n?n==="left"?t.getLeftVisibleLeafColumns():t.getRightVisibleLeafColumns():t.getVisibleLeafColumns(),i=r.findIndex(o=>o.id===e.id);if(i>0){const o=r[i-1];return o.getStart(n)+o.getSize()}return 0},resetSize:()=>{t.setColumnSizing(n=>{let{[e.id]:r,...i}=n;return i})},getCanResize:()=>{var n,r;return((n=e.columnDef.enableResizing)!=null?n:!0)&&((r=t.options.enableColumnResizing)!=null?r:!0)},getIsResizing:()=>t.getState().columnSizingInfo.isResizingColumn===e.id}),createHeader:(e,t)=>({getSize:()=>{let n=0;const r=i=>{if(i.subHeaders.length)i.subHeaders.forEach(r);else{var o;n+=(o=i.column.getSize())!=null?o:0}};return r(e),n},getStart:()=>{if(e.index>0){const n=e.headerGroup.headers[e.index-1];return n.getStart()+n.getSize()}return 0},getResizeHandler:()=>{const n=t.getColumn(e.column.id),r=n==null?void 0:n.getCanResize();return i=>{if(!n||!r||(i.persist==null||i.persist(),Or(i)&&i.touches&&i.touches.length>1))return;const o=e.getSize(),l=e?e.getLeafHeaders().map(m=>[m.column.id,m.column.getSize()]):[[n.id,n.getSize()]],s=Or(i)?Math.round(i.touches[0].clientX):i.clientX,a={},u=(m,_)=>{typeof _=="number"&&(t.setColumnSizingInfo(b=>{var v,$;const A=_-((v=b==null?void 0:b.startOffset)!=null?v:0),k=Math.max(A/(($=b==null?void 0:b.startSize)!=null?$:0),-.999999);return b.columnSizingStart.forEach(E=>{let[y,C]=E;a[y]=Math.round(Math.max(C+C*k,0)*100)/100}),{...b,deltaOffset:A,deltaPercentage:k}}),(t.options.columnResizeMode==="onChange"||m==="end")&&t.setColumnSizing(b=>({...b,...a})))},f=m=>u("move",m),c=m=>{u("end",m),t.setColumnSizingInfo(_=>({..._,isResizingColumn:!1,startOffset:null,startSize:null,deltaOffset:null,deltaPercentage:null,columnSizingStart:[]}))},g={moveHandler:m=>f(m.clientX),upHandler:m=>{document.removeEventListener("mousemove",g.moveHandler),document.removeEventListener("mouseup",g.upHandler),c(m.clientX)}},d={moveHandler:m=>(m.cancelable&&(m.preventDefault(),m.stopPropagation()),f(m.touches[0].clientX),!1),upHandler:m=>{var _;document.removeEventListener("touchmove",d.moveHandler),document.removeEventListener("touchend",d.upHandler),m.cancelable&&(m.preventDefault(),m.stopPropagation()),c((_=m.touches[0])==null?void 0:_.clientX)}},p=Kg()?{passive:!1}:!1;Or(i)?(document.addEventListener("touchmove",d.moveHandler,p),document.addEventListener("touchend",d.upHandler,p)):(document.addEventListener("mousemove",g.moveHandler,p),document.addEventListener("mouseup",g.upHandler,p)),t.setColumnSizingInfo(m=>({...m,startOffset:s,startSize:o,deltaOffset:0,deltaPercentage:0,columnSizingStart:l,isResizingColumn:n.id}))}}}),createTable:e=>({setColumnSizing:t=>e.options.onColumnSizingChange==null?void 0:e.options.onColumnSizingChange(t),setColumnSizingInfo:t=>e.options.onColumnSizingInfoChange==null?void 0:e.options.onColumnSizingInfoChange(t),resetColumnSizing:t=>{var n;e.setColumnSizing(t?{}:(n=e.initialState.columnSizing)!=null?n:{})},resetHeaderSizeInfo:t=>{var n;e.setColumnSizingInfo(t?Nr():(n=e.initialState.columnSizingInfo)!=null?n:Nr())},getTotalSize:()=>{var t,n;return(t=(n=e.getHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0},getLeftTotalSize:()=>{var t,n;return(t=(n=e.getLeftHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0},getCenterTotalSize:()=>{var t,n;return(t=(n=e.getCenterHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0},getRightTotalSize:()=>{var t,n;return(t=(n=e.getRightHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0}})};let Pn=null;function Kg(){if(typeof Pn=="boolean")return Pn;let e=!1;try{const t={get passive(){return e=!0,!1}},n=()=>{};window.addEventListener("test",n,t),window.removeEventListener("test",n)}catch{e=!1}return Pn=e,Pn}function Or(e){return e.type==="touchstart"}const Jg={getInitialState:e=>({expanded:{},...e}),getDefaultOptions:e=>({onExpandedChange:Ge("expanded",e),paginateExpandedRows:!0}),createTable:e=>{let t=!1,n=!1;return{_autoResetExpanded:()=>{var r,i;if(!t){e._queue(()=>{t=!0});return}if((r=(i=e.options.autoResetAll)!=null?i:e.options.autoResetExpanded)!=null?r:!e.options.manualExpanding){if(n)return;n=!0,e._queue(()=>{e.resetExpanded(),n=!1})}},setExpanded:r=>e.options.onExpandedChange==null?void 0:e.options.onExpandedChange(r),toggleAllRowsExpanded:r=>{r??!e.getIsAllRowsExpanded()?e.setExpanded(!0):e.setExpanded({})},resetExpanded:r=>{var i,o;e.setExpanded(r?{}:(i=(o=e.initialState)==null?void 0:o.expanded)!=null?i:{})},getCanSomeRowsExpand:()=>e.getPrePaginationRowModel().flatRows.some(r=>r.getCanExpand()),getToggleAllRowsExpandedHandler:()=>r=>{r.persist==null||r.persist(),e.toggleAllRowsExpanded()},getIsSomeRowsExpanded:()=>{const r=e.getState().expanded;return r===!0||Object.values(r).some(Boolean)},getIsAllRowsExpanded:()=>{const r=e.getState().expanded;return typeof r=="boolean"?r===!0:!(!Object.keys(r).length||e.getRowModel().flatRows.some(i=>!i.getIsExpanded()))},getExpandedDepth:()=>{let r=0;return(e.getState().expanded===!0?Object.keys(e.getRowModel().rowsById):Object.keys(e.getState().expanded)).forEach(o=>{const l=o.split(".");r=Math.max(r,l.length)}),r},getPreExpandedRowModel:()=>e.getSortedRowModel(),getExpandedRowModel:()=>(!e._getExpandedRowModel&&e.options.getExpandedRowModel&&(e._getExpandedRowModel=e.options.getExpandedRowModel(e)),e.options.manualExpanding||!e._getExpandedRowModel?e.getPreExpandedRowModel():e._getExpandedRowModel())}},createRow:(e,t)=>({toggleExpanded:n=>{t.setExpanded(r=>{var i;const o=r===!0?!0:!!(r!=null&&r[e.id]);let l={};if(r===!0?Object.keys(t.getRowModel().rowsById).forEach(s=>{l[s]=!0}):l=r,n=(i=n)!=null?i:!o,!o&&n)return{...l,[e.id]:!0};if(o&&!n){const{[e.id]:s,...a}=l;return a}return r})},getIsExpanded:()=>{var n;const r=t.getState().expanded;return!!((n=t.options.getIsRowExpanded==null?void 0:t.options.getIsRowExpanded(e))!=null?n:r===!0||r!=null&&r[e.id])},getCanExpand:()=>{var n,r,i;return(n=t.options.getRowCanExpand==null?void 0:t.options.getRowCanExpand(e))!=null?n:((r=t.options.enableExpanding)!=null?r:!0)&&!!((i=e.subRows)!=null&&i.length)},getToggleExpandedHandler:()=>{const n=e.getCanExpand();return()=>{n&&e.toggleExpanded()}}})},qs=(e,t,n)=>{var r,i,o;const l=n.toLowerCase();return!!(!((r=e.getValue(t))==null||(i=r.toString())==null||(o=i.toLowerCase())==null)&&o.includes(l))};qs.autoRemove=e=>Xe(e);const Gs=(e,t,n)=>{var r,i;return!!(!((r=e.getValue(t))==null||(i=r.toString())==null)&&i.includes(n))};Gs.autoRemove=e=>Xe(e);const Bs=(e,t,n)=>{var r,i;return((r=e.getValue(t))==null||(i=r.toString())==null?void 0:i.toLowerCase())===(n==null?void 0:n.toLowerCase())};Bs.autoRemove=e=>Xe(e);const js=(e,t,n)=>{var r;return(r=e.getValue(t))==null?void 0:r.includes(n)};js.autoRemove=e=>Xe(e)||!(e!=null&&e.length);const Us=(e,t,n)=>!n.some(r=>{var i;return!((i=e.getValue(t))!=null&&i.includes(r))});Us.autoRemove=e=>Xe(e)||!(e!=null&&e.length);const Ws=(e,t,n)=>n.some(r=>{var i;return(i=e.getValue(t))==null?void 0:i.includes(r)});Ws.autoRemove=e=>Xe(e)||!(e!=null&&e.length);const Xs=(e,t,n)=>e.getValue(t)===n;Xs.autoRemove=e=>Xe(e);const Ys=(e,t,n)=>e.getValue(t)==n;Ys.autoRemove=e=>Xe(e);const Ei=(e,t,n)=>{let[r,i]=n;const o=e.getValue(t);return o>=r&&o<=i};Ei.resolveFilterValue=e=>{let[t,n]=e,r=typeof t!="number"?parseFloat(t):t,i=typeof n!="number"?parseFloat(n):n,o=t===null||Number.isNaN(r)?-1/0:r,l=n===null||Number.isNaN(i)?1/0:i;if(o>l){const s=o;o=l,l=s}return[o,l]};Ei.autoRemove=e=>Xe(e)||Xe(e[0])&&Xe(e[1]);const nt={includesString:qs,includesStringSensitive:Gs,equalsString:Bs,arrIncludes:js,arrIncludesAll:Us,arrIncludesSome:Ws,equals:Xs,weakEquals:Ys,inNumberRange:Ei};function Xe(e){return e==null||e===""}const Zg={getDefaultColumnDef:()=>({filterFn:"auto"}),getInitialState:e=>({columnFilters:[],globalFilter:void 0,...e}),getDefaultOptions:e=>({onColumnFiltersChange:Ge("columnFilters",e),onGlobalFilterChange:Ge("globalFilter",e),filterFromLeafRows:!1,maxLeafRowFilterDepth:100,globalFilterFn:"auto",getColumnCanGlobalFilter:t=>{var n,r;const i=(n=e.getCoreRowModel().flatRows[0])==null||(r=n._getAllCellsByColumnId()[t.id])==null?void 0:r.getValue();return typeof i=="string"||typeof i=="number"}}),createColumn:(e,t)=>({getAutoFilterFn:()=>{const n=t.getCoreRowModel().flatRows[0],r=n==null?void 0:n.getValue(e.id);return typeof r=="string"?nt.includesString:typeof r=="number"?nt.inNumberRange:typeof r=="boolean"||r!==null&&typeof r=="object"?nt.equals:Array.isArray(r)?nt.arrIncludes:nt.weakEquals},getFilterFn:()=>{var n,r;return ar(e.columnDef.filterFn)?e.columnDef.filterFn:e.columnDef.filterFn==="auto"?e.getAutoFilterFn():(n=(r=t.options.filterFns)==null?void 0:r[e.columnDef.filterFn])!=null?n:nt[e.columnDef.filterFn]},getCanFilter:()=>{var n,r,i;return((n=e.columnDef.enableColumnFilter)!=null?n:!0)&&((r=t.options.enableColumnFilters)!=null?r:!0)&&((i=t.options.enableFilters)!=null?i:!0)&&!!e.accessorFn},getCanGlobalFilter:()=>{var n,r,i,o;return((n=e.columnDef.enableGlobalFilter)!=null?n:!0)&&((r=t.options.enableGlobalFilter)!=null?r:!0)&&((i=t.options.enableFilters)!=null?i:!0)&&((o=t.options.getColumnCanGlobalFilter==null?void 0:t.options.getColumnCanGlobalFilter(e))!=null?o:!0)&&!!e.accessorFn},getIsFiltered:()=>e.getFilterIndex()>-1,getFilterValue:()=>{var n,r;return(n=t.getState().columnFilters)==null||(r=n.find(i=>i.id===e.id))==null?void 0:r.value},getFilterIndex:()=>{var n,r;return(n=(r=t.getState().columnFilters)==null?void 0:r.findIndex(i=>i.id===e.id))!=null?n:-1},setFilterValue:n=>{t.setColumnFilters(r=>{const i=e.getFilterFn(),o=r==null?void 0:r.find(f=>f.id===e.id),l=pt(n,o?o.value:void 0);if(cl(i,l,e)){var s;return(s=r==null?void 0:r.filter(f=>f.id!==e.id))!=null?s:[]}const a={id:e.id,value:l};if(o){var u;return(u=r==null?void 0:r.map(f=>f.id===e.id?a:f))!=null?u:[]}return r!=null&&r.length?[...r,a]:[a]})},_getFacetedRowModel:t.options.getFacetedRowModel&&t.options.getFacetedRowModel(t,e.id),getFacetedRowModel:()=>e._getFacetedRowModel?e._getFacetedRowModel():t.getPreFilteredRowModel(),_getFacetedUniqueValues:t.options.getFacetedUniqueValues&&t.options.getFacetedUniqueValues(t,e.id),getFacetedUniqueValues:()=>e._getFacetedUniqueValues?e._getFacetedUniqueValues():new Map,_getFacetedMinMaxValues:t.options.getFacetedMinMaxValues&&t.options.getFacetedMinMaxValues(t,e.id),getFacetedMinMaxValues:()=>{if(e._getFacetedMinMaxValues)return e._getFacetedMinMaxValues()}}),createRow:(e,t)=>({columnFilters:{},columnFiltersMeta:{}}),createTable:e=>({getGlobalAutoFilterFn:()=>nt.includesString,getGlobalFilterFn:()=>{var t,n;const{globalFilterFn:r}=e.options;return ar(r)?r:r==="auto"?e.getGlobalAutoFilterFn():(t=(n=e.options.filterFns)==null?void 0:n[r])!=null?t:nt[r]},setColumnFilters:t=>{const n=e.getAllLeafColumns(),r=i=>{var o;return(o=pt(t,i))==null?void 0:o.filter(l=>{const s=n.find(a=>a.id===l.id);if(s){const a=s.getFilterFn();if(cl(a,l.value,s))return!1}return!0})};e.options.onColumnFiltersChange==null||e.options.onColumnFiltersChange(r)},setGlobalFilter:t=>{e.options.onGlobalFilterChange==null||e.options.onGlobalFilterChange(t)},resetGlobalFilter:t=>{e.setGlobalFilter(t?void 0:e.initialState.globalFilter)},resetColumnFilters:t=>{var n,r;e.setColumnFilters(t?[]:(n=(r=e.initialState)==null?void 0:r.columnFilters)!=null?n:[])},getPreFilteredRowModel:()=>e.getCoreRowModel(),getFilteredRowModel:()=>(!e._getFilteredRowModel&&e.options.getFilteredRowModel&&(e._getFilteredRowModel=e.options.getFilteredRowModel(e)),e.options.manualFiltering||!e._getFilteredRowModel?e.getPreFilteredRowModel():e._getFilteredRowModel()),_getGlobalFacetedRowModel:e.options.getFacetedRowModel&&e.options.getFacetedRowModel(e,"__global__"),getGlobalFacetedRowModel:()=>e.options.manualFiltering||!e._getGlobalFacetedRowModel?e.getPreFilteredRowModel():e._getGlobalFacetedRowModel(),_getGlobalFacetedUniqueValues:e.options.getFacetedUniqueValues&&e.options.getFacetedUniqueValues(e,"__global__"),getGlobalFacetedUniqueValues:()=>e._getGlobalFacetedUniqueValues?e._getGlobalFacetedUniqueValues():new Map,_getGlobalFacetedMinMaxValues:e.options.getFacetedMinMaxValues&&e.options.getFacetedMinMaxValues(e,"__global__"),getGlobalFacetedMinMaxValues:()=>{if(e._getGlobalFacetedMinMaxValues)return e._getGlobalFacetedMinMaxValues()}})};function cl(e,t,n){return(e&&e.autoRemove?e.autoRemove(t,n):!1)||typeof t>"u"||typeof t=="string"&&!t}const eh=(e,t,n)=>n.reduce((r,i)=>{const o=i.getValue(e);return r+(typeof o=="number"?o:0)},0),th=(e,t,n)=>{let r;return n.forEach(i=>{const o=i.getValue(e);o!=null&&(r>o||r===void 0&&o>=o)&&(r=o)}),r},nh=(e,t,n)=>{let r;return n.forEach(i=>{const o=i.getValue(e);o!=null&&(r {u|=1< e;const n=typeof e;if(n!==typeof t||Array.isArray(e)!==Array.isArray(t))throw new Error("Cannot interpolate values of different type");if(Array.isArray(e)){const r=t.map((i,o)=>Wr(e[o],i));return i=>r.map(o=>o(i))}if(n==="object"){if(!e||!t)throw new Error("Object cannot be null");if(xi(e)&&xi(t)){e=e.getTime(),t=t.getTime();const o=t-e;return l=>new Date(e+l*o)}const r=Object.keys(t),i={};return r.forEach(o=>{i[o]=Wr(e[o],t[o])}),o=>{const l={};return r.forEach(a=>{l[a]=i[a](o)}),l}}if(n==="number"){const r=t-e;return i=>e+i*r}throw new Error(`Cannot interpolate ${n} values`)}function mi(e,t={}){const n=ke(e);let r,i=e;function o(l,a){if(e==null)return n.set(e=l),Promise.resolve();i=l;let s=r,u=!1,{delay:c=0,duration:f=400,easing:d=ts,interpolate:g=Wr}=Bn(Bn({},t),a);if(f===0)return s&&(s.abort(),s=null),n.set(e=i),Promise.resolve();const p=is()+c;let m;return r=ls(_=>{if(_ f?(n.set(e=l),!1):(n.set(e=m(d(b/f))),!0)}),r.promise}return{set:o,update:(l,a)=>o(l(i,e),a),subscribe:n.subscribe}}const xe=ke([]),Cr=ke([]),kr=ke([]),pn=ke([]),nt=ke("all"),Ti=ke({"n/a":1,up:1,down:1}),lu=["who","what","how","why","where","does","can","n/a"],Xr=mi(lu.map(e=>({question:e,count:0}))),Yr=ke(new Set);var Qr="http://www.w3.org/1999/xhtml";const Oi={svg:"http://www.w3.org/2000/svg",xhtml:Qr,xlink:"http://www.w3.org/1999/xlink",xml:"http://www.w3.org/XML/1998/namespace",xmlns:"http://www.w3.org/2000/xmlns/"};function hs(e){var t=e+="",n=t.indexOf(":");return n>=0&&(t=e.slice(0,n))!=="xmlns"&&(e=e.slice(n+1)),Oi.hasOwnProperty(t)?{space:Oi[t],local:e}:e}function su(e){return function(){var t=this.ownerDocument,n=this.namespaceURI;return n===Qr&&t.documentElement.namespaceURI===Qr?t.createElement(e):t.createElementNS(n,e)}}function au(e){return function(){return this.ownerDocument.createElementNS(e.space,e.local)}}function ps(e){var t=hs(e);return(t.local?au:su)(t)}function uu(){}function ms(e){return e==null?uu:function(){return this.querySelector(e)}}function cu(e){typeof e!="function"&&(e=ms(e));for(var t=this._groups,n=t.length,r=new Array(n),i=0;i =0;)(l=r[i])&&(o&&l.compareDocumentPosition(o)^4&&o.parentNode.insertBefore(l,o),o=l);return this}function Tu(e){e||(e=Ou);function t(f,d){return f&&d?e(f.__data__,d.__data__):!f-!d}for(var n=this._groups,r=n.length,i=new Array(r),o=0;o Ask a question to receive an answer from the chatbot. If the response is
+ satisfactory, click on the Ask a question and click on the better of the two responses. The better
+ response will be outlined in green, the worse
+ response outlined in red. This data will be
+ automatically fed to RLHF. Ask a question to receive an answer from the chatbot. If the response is
+ satisfactory, click on the 0?g[o-1]:f,w.x1=o Ask a question and rank the answers across the models. Drag each answer to
+ rank it, in ascending order, from left-to-right. Optionally, select the
+ rank for each via the corresponding dropdown. {let g=d;for(const m of s.split(".")){var p;g=(p=g)==null?void 0:p[m]}return g}:c=d=>d[a.accessorKey]),!u)throw new Error;let f={id:`${String(u)}`,accessorFn:c,parent:r,depth:n,columnDef:a,columns:[],getFlatColumns:G(()=>[!0],()=>{var d;return[f,...(d=f.columns)==null?void 0:d.flatMap(g=>g.getFlatColumns())]},{key:"column.getFlatColumns",debug:()=>{var d;return(d=e.options.debugAll)!=null?d:e.options.debugColumns}}),getLeafColumns:G(()=>[e._getOrderColumnsFn()],d=>{var g;if((g=f.columns)!=null&&g.length){let p=f.columns.flatMap(m=>m.getLeafColumns());return d(p)}return[f]},{key:"column.getLeafColumns",debug:()=>{var d;return(d=e.options.debugAll)!=null?d:e.options.debugColumns}})};return f=e._features.reduce((d,g)=>Object.assign(d,g.createColumn==null?void 0:g.createColumn(f,e)),f),f}function $l(e,t,n){var r;let o={id:(r=n.id)!=null?r:t.id,column:t,index:n.index,isPlaceholder:!!n.isPlaceholder,placeholderId:n.placeholderId,depth:n.depth,subHeaders:[],colSpan:0,rowSpan:0,headerGroup:null,getLeafHeaders:()=>{const l=[],a=s=>{s.subHeaders&&s.subHeaders.length&&s.subHeaders.map(a),l.push(s)};return a(o),l},getContext:()=>({table:e,header:o,column:t})};return e._features.forEach(l=>{Object.assign(o,l.createHeader==null?void 0:l.createHeader(o,e))}),o}const Vh={createTable:e=>({getHeaderGroups:G(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.left,e.getState().columnPinning.right],(t,n,r,i)=>{var o,l;const a=(o=r==null?void 0:r.map(f=>n.find(d=>d.id===f)).filter(Boolean))!=null?o:[],s=(l=i==null?void 0:i.map(f=>n.find(d=>d.id===f)).filter(Boolean))!=null?l:[],u=n.filter(f=>!(r!=null&&r.includes(f.id))&&!(i!=null&&i.includes(f.id)));return xn(t,[...a,...u,...s],e)},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterHeaderGroups:G(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.left,e.getState().columnPinning.right],(t,n,r,i)=>(n=n.filter(o=>!(r!=null&&r.includes(o.id))&&!(i!=null&&i.includes(o.id))),xn(t,n,e,"center")),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftHeaderGroups:G(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.left],(t,n,r)=>{var i;const o=(i=r==null?void 0:r.map(l=>n.find(a=>a.id===l)).filter(Boolean))!=null?i:[];return xn(t,o,e,"left")},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightHeaderGroups:G(()=>[e.getAllColumns(),e.getVisibleLeafColumns(),e.getState().columnPinning.right],(t,n,r)=>{var i;const o=(i=r==null?void 0:r.map(l=>n.find(a=>a.id===l)).filter(Boolean))!=null?i:[];return xn(t,o,e,"right")},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getFooterGroups:G(()=>[e.getHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftFooterGroups:G(()=>[e.getLeftHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterFooterGroups:G(()=>[e.getCenterHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightFooterGroups:G(()=>[e.getRightHeaderGroups()],t=>[...t].reverse(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getFlatHeaders:G(()=>[e.getHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftFlatHeaders:G(()=>[e.getLeftHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterFlatHeaders:G(()=>[e.getCenterHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightFlatHeaders:G(()=>[e.getRightHeaderGroups()],t=>t.map(n=>n.headers).flat(),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getCenterLeafHeaders:G(()=>[e.getCenterFlatHeaders()],t=>t.filter(n=>{var r;return!((r=n.subHeaders)!=null&&r.length)}),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeftLeafHeaders:G(()=>[e.getLeftFlatHeaders()],t=>t.filter(n=>{var r;return!((r=n.subHeaders)!=null&&r.length)}),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getRightLeafHeaders:G(()=>[e.getRightFlatHeaders()],t=>t.filter(n=>{var r;return!((r=n.subHeaders)!=null&&r.length)}),{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}}),getLeafHeaders:G(()=>[e.getLeftHeaderGroups(),e.getCenterHeaderGroups(),e.getRightHeaderGroups()],(t,n,r)=>{var i,o,l,a,s,u;return[...(i=(o=t[0])==null?void 0:o.headers)!=null?i:[],...(l=(a=n[0])==null?void 0:a.headers)!=null?l:[],...(s=(u=r[0])==null?void 0:u.headers)!=null?s:[]].map(c=>c.getLeafHeaders()).flat()},{key:!1,debug:()=>{var t;return(t=e.options.debugAll)!=null?t:e.options.debugHeaders}})})};function xn(e,t,n,r){var i,o;let l=0;const a=function(d,g){g===void 0&&(g=1),l=Math.max(l,g),d.filter(p=>p.getIsVisible()).forEach(p=>{var m;(m=p.columns)!=null&&m.length&&a(p.columns,g+1)},0)};a(e);let s=[];const u=(d,g)=>{const p={depth:g,id:[r,`${g}`].filter(Boolean).join("_"),headers:[]},m=[];d.forEach(_=>{const b=[...m].reverse()[0],w=_.column.depth===p.depth;let C,A=!1;if(w&&_.column.parent?C=_.column.parent:(C=_.column,A=!0),b&&(b==null?void 0:b.column)===C)b.subHeaders.push(_);else{const $=$l(n,C,{id:[r,g,C.id,_==null?void 0:_.id].filter(Boolean).join("_"),isPlaceholder:A,placeholderId:A?`${m.filter(E=>E.column===C).length}`:void 0,depth:g,index:m.length});$.subHeaders.push(_),m.push($)}p.headers.push(_),_.headerGroup=p}),s.push(p),g>0&&u(m,g-1)},c=t.map((d,g)=>$l(n,d,{depth:l,index:g}));u(c,l-1),s.reverse();const f=d=>d.filter(p=>p.column.getIsVisible()).map(p=>{let m=0,_=0,b=[0];p.subHeaders&&p.subHeaders.length?(b=[],f(p.subHeaders).forEach(C=>{let{colSpan:A,rowSpan:$}=C;m+=A,b.push($)})):m=1;const w=Math.min(...b);return _=_+w,p.colSpan=m,p.rowSpan=_,{colSpan:m,rowSpan:_}});return f((i=(o=s[0])==null?void 0:o.headers)!=null?i:[]),s}const Tn={size:150,minSize:20,maxSize:Number.MAX_SAFE_INTEGER},Vr=()=>({startOffset:null,startSize:null,deltaOffset:null,deltaPercentage:null,isResizingColumn:!1,columnSizingStart:[]}),zh={getDefaultColumnDef:()=>Tn,getInitialState:e=>({columnSizing:{},columnSizingInfo:Vr(),...e}),getDefaultOptions:e=>({columnResizeMode:"onEnd",onColumnSizingChange:je("columnSizing",e),onColumnSizingInfoChange:je("columnSizingInfo",e)}),createColumn:(e,t)=>({getSize:()=>{var n,r,i;const o=t.getState().columnSizing[e.id];return Math.min(Math.max((n=e.columnDef.minSize)!=null?n:Tn.minSize,(r=o??e.columnDef.size)!=null?r:Tn.size),(i=e.columnDef.maxSize)!=null?i:Tn.maxSize)},getStart:n=>{const r=n?n==="left"?t.getLeftVisibleLeafColumns():t.getRightVisibleLeafColumns():t.getVisibleLeafColumns(),i=r.findIndex(o=>o.id===e.id);if(i>0){const o=r[i-1];return o.getStart(n)+o.getSize()}return 0},resetSize:()=>{t.setColumnSizing(n=>{let{[e.id]:r,...i}=n;return i})},getCanResize:()=>{var n,r;return((n=e.columnDef.enableResizing)!=null?n:!0)&&((r=t.options.enableColumnResizing)!=null?r:!0)},getIsResizing:()=>t.getState().columnSizingInfo.isResizingColumn===e.id}),createHeader:(e,t)=>({getSize:()=>{let n=0;const r=i=>{if(i.subHeaders.length)i.subHeaders.forEach(r);else{var o;n+=(o=i.column.getSize())!=null?o:0}};return r(e),n},getStart:()=>{if(e.index>0){const n=e.headerGroup.headers[e.index-1];return n.getStart()+n.getSize()}return 0},getResizeHandler:()=>{const n=t.getColumn(e.column.id),r=n==null?void 0:n.getCanResize();return i=>{if(!n||!r||(i.persist==null||i.persist(),zr(i)&&i.touches&&i.touches.length>1))return;const o=e.getSize(),l=e?e.getLeafHeaders().map(m=>[m.column.id,m.column.getSize()]):[[n.id,n.getSize()]],a=zr(i)?Math.round(i.touches[0].clientX):i.clientX,s={},u=(m,_)=>{typeof _=="number"&&(t.setColumnSizingInfo(b=>{var w,C;const A=_-((w=b==null?void 0:b.startOffset)!=null?w:0),$=Math.max(A/((C=b==null?void 0:b.startSize)!=null?C:0),-.999999);return b.columnSizingStart.forEach(E=>{let[S,k]=E;s[S]=Math.round(Math.max(k+k*$,0)*100)/100}),{...b,deltaOffset:A,deltaPercentage:$}}),(t.options.columnResizeMode==="onChange"||m==="end")&&t.setColumnSizing(b=>({...b,...s})))},c=m=>u("move",m),f=m=>{u("end",m),t.setColumnSizingInfo(_=>({..._,isResizingColumn:!1,startOffset:null,startSize:null,deltaOffset:null,deltaPercentage:null,columnSizingStart:[]}))},d={moveHandler:m=>c(m.clientX),upHandler:m=>{document.removeEventListener("mousemove",d.moveHandler),document.removeEventListener("mouseup",d.upHandler),f(m.clientX)}},g={moveHandler:m=>(m.cancelable&&(m.preventDefault(),m.stopPropagation()),c(m.touches[0].clientX),!1),upHandler:m=>{var _;document.removeEventListener("touchmove",g.moveHandler),document.removeEventListener("touchend",g.upHandler),m.cancelable&&(m.preventDefault(),m.stopPropagation()),f((_=m.touches[0])==null?void 0:_.clientX)}},p=Hh()?{passive:!1}:!1;zr(i)?(document.addEventListener("touchmove",g.moveHandler,p),document.addEventListener("touchend",g.upHandler,p)):(document.addEventListener("mousemove",d.moveHandler,p),document.addEventListener("mouseup",d.upHandler,p)),t.setColumnSizingInfo(m=>({...m,startOffset:a,startSize:o,deltaOffset:0,deltaPercentage:0,columnSizingStart:l,isResizingColumn:n.id}))}}}),createTable:e=>({setColumnSizing:t=>e.options.onColumnSizingChange==null?void 0:e.options.onColumnSizingChange(t),setColumnSizingInfo:t=>e.options.onColumnSizingInfoChange==null?void 0:e.options.onColumnSizingInfoChange(t),resetColumnSizing:t=>{var n;e.setColumnSizing(t?{}:(n=e.initialState.columnSizing)!=null?n:{})},resetHeaderSizeInfo:t=>{var n;e.setColumnSizingInfo(t?Vr():(n=e.initialState.columnSizingInfo)!=null?n:Vr())},getTotalSize:()=>{var t,n;return(t=(n=e.getHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0},getLeftTotalSize:()=>{var t,n;return(t=(n=e.getLeftHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0},getCenterTotalSize:()=>{var t,n;return(t=(n=e.getCenterHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0},getRightTotalSize:()=>{var t,n;return(t=(n=e.getRightHeaderGroups()[0])==null?void 0:n.headers.reduce((r,i)=>r+i.getSize(),0))!=null?t:0}})};let On=null;function Hh(){if(typeof On=="boolean")return On;let e=!1;try{const t={get passive(){return e=!0,!1}},n=()=>{};window.addEventListener("test",n,t),window.removeEventListener("test",n)}catch{e=!1}return On=e,On}function zr(e){return e.type==="touchstart"}const Lh={getInitialState:e=>({expanded:{},...e}),getDefaultOptions:e=>({onExpandedChange:je("expanded",e),paginateExpandedRows:!0}),createTable:e=>{let t=!1,n=!1;return{_autoResetExpanded:()=>{var r,i;if(!t){e._queue(()=>{t=!0});return}if((r=(i=e.options.autoResetAll)!=null?i:e.options.autoResetExpanded)!=null?r:!e.options.manualExpanding){if(n)return;n=!0,e._queue(()=>{e.resetExpanded(),n=!1})}},setExpanded:r=>e.options.onExpandedChange==null?void 0:e.options.onExpandedChange(r),toggleAllRowsExpanded:r=>{r??!e.getIsAllRowsExpanded()?e.setExpanded(!0):e.setExpanded({})},resetExpanded:r=>{var i,o;e.setExpanded(r?{}:(i=(o=e.initialState)==null?void 0:o.expanded)!=null?i:{})},getCanSomeRowsExpand:()=>e.getPrePaginationRowModel().flatRows.some(r=>r.getCanExpand()),getToggleAllRowsExpandedHandler:()=>r=>{r.persist==null||r.persist(),e.toggleAllRowsExpanded()},getIsSomeRowsExpanded:()=>{const r=e.getState().expanded;return r===!0||Object.values(r).some(Boolean)},getIsAllRowsExpanded:()=>{const r=e.getState().expanded;return typeof r=="boolean"?r===!0:!(!Object.keys(r).length||e.getRowModel().flatRows.some(i=>!i.getIsExpanded()))},getExpandedDepth:()=>{let r=0;return(e.getState().expanded===!0?Object.keys(e.getRowModel().rowsById):Object.keys(e.getState().expanded)).forEach(o=>{const l=o.split(".");r=Math.max(r,l.length)}),r},getPreExpandedRowModel:()=>e.getSortedRowModel(),getExpandedRowModel:()=>(!e._getExpandedRowModel&&e.options.getExpandedRowModel&&(e._getExpandedRowModel=e.options.getExpandedRowModel(e)),e.options.manualExpanding||!e._getExpandedRowModel?e.getPreExpandedRowModel():e._getExpandedRowModel())}},createRow:(e,t)=>({toggleExpanded:n=>{t.setExpanded(r=>{var i;const o=r===!0?!0:!!(r!=null&&r[e.id]);let l={};if(r===!0?Object.keys(t.getRowModel().rowsById).forEach(a=>{l[a]=!0}):l=r,n=(i=n)!=null?i:!o,!o&&n)return{...l,[e.id]:!0};if(o&&!n){const{[e.id]:a,...s}=l;return s}return r})},getIsExpanded:()=>{var n;const r=t.getState().expanded;return!!((n=t.options.getIsRowExpanded==null?void 0:t.options.getIsRowExpanded(e))!=null?n:r===!0||r!=null&&r[e.id])},getCanExpand:()=>{var n,r,i;return(n=t.options.getRowCanExpand==null?void 0:t.options.getRowCanExpand(e))!=null?n:((r=t.options.enableExpanding)!=null?r:!0)&&!!((i=e.subRows)!=null&&i.length)},getToggleExpandedHandler:()=>{const n=e.getCanExpand();return()=>{n&&e.toggleExpanded()}}})},ga=(e,t,n)=>{var r,i,o;const l=n.toLowerCase();return!!(!((r=e.getValue(t))==null||(i=r.toString())==null||(o=i.toLowerCase())==null)&&o.includes(l))};ga.autoRemove=e=>Xe(e);const ha=(e,t,n)=>{var r,i;return!!(!((r=e.getValue(t))==null||(i=r.toString())==null)&&i.includes(n))};ha.autoRemove=e=>Xe(e);const pa=(e,t,n)=>{var r,i;return((r=e.getValue(t))==null||(i=r.toString())==null?void 0:i.toLowerCase())===(n==null?void 0:n.toLowerCase())};pa.autoRemove=e=>Xe(e);const ma=(e,t,n)=>{var r;return(r=e.getValue(t))==null?void 0:r.includes(n)};ma.autoRemove=e=>Xe(e)||!(e!=null&&e.length);const _a=(e,t,n)=>!n.some(r=>{var i;return!((i=e.getValue(t))!=null&&i.includes(r))});_a.autoRemove=e=>Xe(e)||!(e!=null&&e.length);const va=(e,t,n)=>n.some(r=>{var i;return(i=e.getValue(t))==null?void 0:i.includes(r)});va.autoRemove=e=>Xe(e)||!(e!=null&&e.length);const ba=(e,t,n)=>e.getValue(t)===n;ba.autoRemove=e=>Xe(e);const wa=(e,t,n)=>e.getValue(t)==n;wa.autoRemove=e=>Xe(e);const Ei=(e,t,n)=>{let[r,i]=n;const o=e.getValue(t);return o>=r&&o<=i};Ei.resolveFilterValue=e=>{let[t,n]=e,r=typeof t!="number"?parseFloat(t):t,i=typeof n!="number"?parseFloat(n):n,o=t===null||Number.isNaN(r)?-1/0:r,l=n===null||Number.isNaN(i)?1/0:i;if(o>l){const a=o;o=l,l=a}return[o,l]};Ei.autoRemove=e=>Xe(e)||Xe(e[0])&&Xe(e[1]);const it={includesString:ga,includesStringSensitive:ha,equalsString:pa,arrIncludes:ma,arrIncludesAll:_a,arrIncludesSome:va,equals:ba,weakEquals:wa,inNumberRange:Ei};function Xe(e){return e==null||e===""}const qh={getDefaultColumnDef:()=>({filterFn:"auto"}),getInitialState:e=>({columnFilters:[],globalFilter:void 0,...e}),getDefaultOptions:e=>({onColumnFiltersChange:je("columnFilters",e),onGlobalFilterChange:je("globalFilter",e),filterFromLeafRows:!1,maxLeafRowFilterDepth:100,globalFilterFn:"auto",getColumnCanGlobalFilter:t=>{var n,r;const i=(n=e.getCoreRowModel().flatRows[0])==null||(r=n._getAllCellsByColumnId()[t.id])==null?void 0:r.getValue();return typeof i=="string"||typeof i=="number"}}),createColumn:(e,t)=>({getAutoFilterFn:()=>{const n=t.getCoreRowModel().flatRows[0],r=n==null?void 0:n.getValue(e.id);return typeof r=="string"?it.includesString:typeof r=="number"?it.inNumberRange:typeof r=="boolean"||r!==null&&typeof r=="object"?it.equals:Array.isArray(r)?it.arrIncludes:it.weakEquals},getFilterFn:()=>{var n,r;return fr(e.columnDef.filterFn)?e.columnDef.filterFn:e.columnDef.filterFn==="auto"?e.getAutoFilterFn():(n=(r=t.options.filterFns)==null?void 0:r[e.columnDef.filterFn])!=null?n:it[e.columnDef.filterFn]},getCanFilter:()=>{var n,r,i;return((n=e.columnDef.enableColumnFilter)!=null?n:!0)&&((r=t.options.enableColumnFilters)!=null?r:!0)&&((i=t.options.enableFilters)!=null?i:!0)&&!!e.accessorFn},getCanGlobalFilter:()=>{var n,r,i,o;return((n=e.columnDef.enableGlobalFilter)!=null?n:!0)&&((r=t.options.enableGlobalFilter)!=null?r:!0)&&((i=t.options.enableFilters)!=null?i:!0)&&((o=t.options.getColumnCanGlobalFilter==null?void 0:t.options.getColumnCanGlobalFilter(e))!=null?o:!0)&&!!e.accessorFn},getIsFiltered:()=>e.getFilterIndex()>-1,getFilterValue:()=>{var n,r;return(n=t.getState().columnFilters)==null||(r=n.find(i=>i.id===e.id))==null?void 0:r.value},getFilterIndex:()=>{var n,r;return(n=(r=t.getState().columnFilters)==null?void 0:r.findIndex(i=>i.id===e.id))!=null?n:-1},setFilterValue:n=>{t.setColumnFilters(r=>{const i=e.getFilterFn(),o=r==null?void 0:r.find(c=>c.id===e.id),l=pt(n,o?o.value:void 0);if(Rl(i,l,e)){var a;return(a=r==null?void 0:r.filter(c=>c.id!==e.id))!=null?a:[]}const s={id:e.id,value:l};if(o){var u;return(u=r==null?void 0:r.map(c=>c.id===e.id?s:c))!=null?u:[]}return r!=null&&r.length?[...r,s]:[s]})},_getFacetedRowModel:t.options.getFacetedRowModel&&t.options.getFacetedRowModel(t,e.id),getFacetedRowModel:()=>e._getFacetedRowModel?e._getFacetedRowModel():t.getPreFilteredRowModel(),_getFacetedUniqueValues:t.options.getFacetedUniqueValues&&t.options.getFacetedUniqueValues(t,e.id),getFacetedUniqueValues:()=>e._getFacetedUniqueValues?e._getFacetedUniqueValues():new Map,_getFacetedMinMaxValues:t.options.getFacetedMinMaxValues&&t.options.getFacetedMinMaxValues(t,e.id),getFacetedMinMaxValues:()=>{if(e._getFacetedMinMaxValues)return e._getFacetedMinMaxValues()}}),createRow:(e,t)=>({columnFilters:{},columnFiltersMeta:{}}),createTable:e=>({getGlobalAutoFilterFn:()=>it.includesString,getGlobalFilterFn:()=>{var t,n;const{globalFilterFn:r}=e.options;return fr(r)?r:r==="auto"?e.getGlobalAutoFilterFn():(t=(n=e.options.filterFns)==null?void 0:n[r])!=null?t:it[r]},setColumnFilters:t=>{const n=e.getAllLeafColumns(),r=i=>{var o;return(o=pt(t,i))==null?void 0:o.filter(l=>{const a=n.find(s=>s.id===l.id);if(a){const s=a.getFilterFn();if(Rl(s,l.value,a))return!1}return!0})};e.options.onColumnFiltersChange==null||e.options.onColumnFiltersChange(r)},setGlobalFilter:t=>{e.options.onGlobalFilterChange==null||e.options.onGlobalFilterChange(t)},resetGlobalFilter:t=>{e.setGlobalFilter(t?void 0:e.initialState.globalFilter)},resetColumnFilters:t=>{var n,r;e.setColumnFilters(t?[]:(n=(r=e.initialState)==null?void 0:r.columnFilters)!=null?n:[])},getPreFilteredRowModel:()=>e.getCoreRowModel(),getFilteredRowModel:()=>(!e._getFilteredRowModel&&e.options.getFilteredRowModel&&(e._getFilteredRowModel=e.options.getFilteredRowModel(e)),e.options.manualFiltering||!e._getFilteredRowModel?e.getPreFilteredRowModel():e._getFilteredRowModel()),_getGlobalFacetedRowModel:e.options.getFacetedRowModel&&e.options.getFacetedRowModel(e,"__global__"),getGlobalFacetedRowModel:()=>e.options.manualFiltering||!e._getGlobalFacetedRowModel?e.getPreFilteredRowModel():e._getGlobalFacetedRowModel(),_getGlobalFacetedUniqueValues:e.options.getFacetedUniqueValues&&e.options.getFacetedUniqueValues(e,"__global__"),getGlobalFacetedUniqueValues:()=>e._getGlobalFacetedUniqueValues?e._getGlobalFacetedUniqueValues():new Map,_getGlobalFacetedMinMaxValues:e.options.getFacetedMinMaxValues&&e.options.getFacetedMinMaxValues(e,"__global__"),getGlobalFacetedMinMaxValues:()=>{if(e._getGlobalFacetedMinMaxValues)return e._getGlobalFacetedMinMaxValues()}})};function Rl(e,t,n){return(e&&e.autoRemove?e.autoRemove(t,n):!1)||typeof t>"u"||typeof t=="string"&&!t}const Gh=(e,t,n)=>n.reduce((r,i)=>{const o=i.getValue(e);return r+(typeof o=="number"?o:0)},0),jh=(e,t,n)=>{let r;return n.forEach(i=>{const o=i.getValue(e);o!=null&&(r>o||r===void 0&&o>=o)&&(r=o)}),r},Bh=(e,t,n)=>{let r;return n.forEach(i=>{const o=i.getValue(e);o!=null&&(r {message.question}
- âšī¸ Retrieval Sources
- đ Source: {message.source} ⌃ ⌄ {message.source_content} {i + 1}: {source} ⌃ ⌄ {source_content}
+ Ask a question to receive an answer from the chatbot. If the response is
+ satisfactory, click on the 0))return[];if(e===t)return[e];const r=t0?(e=Math.floor(e/i)*i,t=Math.ceil(t/i)*i):i<0&&(e=Math.ceil(e*i)/i,t=Math.floor(t*i)/i),r=i}}function lc(e){return Math.max(1,Math.ceil(Math.log(Wf(e))/Math.LN2)+1)}function ps(){var e=Kf,t=sn,n=lc;function r(i){Array.isArray(i)||(i=Array.from(i));var o,l=i.length,s,a,u=new Array(l);for(o=0;o0;)--s}i(e[n],o)===0?Kt(e,n,s):(++s,Kt(e,s,r)),s<=t&&(n=s+1),t<=s&&(r=s-1)}return e}function Kt(e,t,n){const r=e[t];e[t]=e[n],e[n]=r}function sc(e,t,n){if(e=Float64Array.from(Bf(e,n)),!(!(r=e.length)||isNaN(t=+t))){if(t<=0||r<2)return Zi(e);if(t>=1)return kt(e);var r,i=(r-1)*t,o=Math.floor(i),l=kt(ms(e,o).subarray(0,o+1)),s=Zi(e.subarray(o+1));return l+(s-l)*(i-o)}}function ac(e,t,n){e=+e,t=+t,n=(i=arguments.length)<2?(t=e,e=0,1):i<3?1:+n;for(var r=-1,i=Math.max(0,Math.ceil((t-e)/n))|0,o=new Array(i);++r=1e21?e.toLocaleString("en").replace(/,/g,""):e.toString(10)}function Yn(e,t){if((n=(e=t?e.toExponential(t-1):e.toExponential()).indexOf("e"))<0)return null;var n,r=e.slice(0,n);return[r.length>1?r[0]+r.slice(2):r,+e.slice(n+1)]}function jt(e){return e=Yn(Math.abs(e)),e?e[1]:NaN}function fc(e,t){return function(n,r){for(var i=n.length,o=[],l=0,s=e[0],a=0;i>0&&s>0&&(a+s+1>r&&(s=Math.max(1,r-a)),o.push(n.substring(i-=s,i+s)),!((a+=s+1)>r));)s=e[l=(l+1)%e.length];return o.reverse().join(t)}}function cc(e){return function(t){return t.replace(/[0-9]/g,function(n){return e[+n]})}}var dc=/^(?:(.)?([<>=^]))?([+\-( ])?([$#])?(0)?(\d+)?(,)?(\.\d+)?(~)?([a-z%])?$/i;function Qn(e){if(!(t=dc.exec(e)))throw new Error("invalid format: "+e);var t;return new wi({fill:t[1],align:t[2],sign:t[3],symbol:t[4],zero:t[5],width:t[6],comma:t[7],precision:t[8]&&t[8].slice(1),trim:t[9],type:t[10]})}Qn.prototype=wi.prototype;function wi(e){this.fill=e.fill===void 0?" ":e.fill+"",this.align=e.align===void 0?">":e.align+"",this.sign=e.sign===void 0?"-":e.sign+"",this.symbol=e.symbol===void 0?"":e.symbol+"",this.zero=!!e.zero,this.width=e.width===void 0?void 0:+e.width,this.comma=!!e.comma,this.precision=e.precision===void 0?void 0:+e.precision,this.trim=!!e.trim,this.type=e.type===void 0?"":e.type+""}wi.prototype.toString=function(){return this.fill+this.align+this.sign+this.symbol+(this.zero?"0":"")+(this.width===void 0?"":Math.max(1,this.width|0))+(this.comma?",":"")+(this.precision===void 0?"":"."+Math.max(0,this.precision|0))+(this.trim?"~":"")+this.type};function gc(e){e:for(var t=e.length,n=1,r=-1,i;n
-
- `,r=P(),i=R("div"),o=R("section"),l=R("div");for(let E=0;E
+
+ `,r=D(),i=R("div"),o=R("section"),l=R("div");for(let E=0;EQuestion:
â ī¸ No Retrieval Sources selected
+ {:else if sources[0] === "Loading..."}
+ đ Loading...
+ {:else if sources[0] === "Not loaded"}
+ â ī¸ No sources loaded
+ {:else}
+ đ Response Sources ({sources.length})
+ {#if show_content}
+ ⌃
+ {:else}
+ ⌄
+ {/if}
+ Vote Feedback Instructions
+
+ {#each items as item}
+
+{#each items as item}
+ {#if activeTabValue == item.value}
+ {item.label}
+
+