feat(demo): add a Gradio demo for the code chat (#124)

* feat(demo): add Gradio chat demo

* feat(demo): improve UI of chat demo

* refactor(demo): update folder org

* feat(demo): made auth UI optional

* feat(demo): add a few examples

* docs(readme): add GIF of demo

* docs(contributing): update codebase description

* docs(readme): add poetry to installation steps

* build(deps-dev): add demo deps

* ci(demo): add job to check the demo

* ci(labeler): add entry for demo

* ci(demo): fix workflow

* ci(demo): debug

* ci(demo): simplify CI job

* fix(demo): fix assets path

.github/labeler.yml

@@ -10,6 +10,10 @@
 - changed-files:
   - any-glob-to-any-file: docker/apm/*
 
+'ext: demo':
+- changed-files:
+  - any-glob-to-any-file: demo/*
+
 'module: core':
 - changed-files:
   - any-glob-to-any-file:

.github/workflows/demo.yml

@@ -0,0 +1,35 @@
+name: demo
+
+on:
+  push:
+    branches: main
+  pull_request:
+    branches: main
+
+jobs:
+  gradio:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.9"
+      - uses: abatilo/actions-poetry@v3
+        with:
+          poetry-version: "1.7.1"
+      - name: Resolve dependencies
+        run: poetry export -f requirements.txt --without-hashes --only demo --output demo/requirements.txt
+      - name: Install demo dependencies
+        run: |
+          python -m pip install --upgrade uv
+          uv pip install --system -r demo/requirements.txt
+      - name: Run & check demo
+        env:
+          SUPERADMIN_LOGIN: dummy_login
+          SUPERADMIN_PWD: dummy_pwd
+        run: |
+          sleep 10 && screen -dm python demo/main.py --auth --port 8080
+          sleep 5 && nc -vz localhost 8080

CONTRIBUTING.md

@@ -14,6 +14,7 @@ Whatever the way you wish to contribute to the project, please respect the [code
 - [`docker`](https://github.com/quack-ai/contribution-api/blob/main/docker) - Docker-related configurations
 - [`docs`](https://github.com/quack-ai/contribution-api/blob/main/docs) - Everything related to documentation
 - [`scripts`](https://github.com/quack-ai/contribution-api/blob/main/scripts) - Custom scripts
+- [`demo`](https://github.com/quack-ai/contribution-api/blob/main/demo) - Code for the Gradio demo
 
 
 ## Continuous Integration

README.md

@@ -45,22 +45,17 @@
 
 ## Quick Tour
 
-### Running/stopping the service
+### Code chat endpoint
 
-You can run the API containers using this command:
+![Code chat demo](https://github.com/quack-ai/platform/releases/download/v0.0.1/companion_demo.gif)
 
-```shell
-make run
-```
+The backend API is the gatekeeper for your LLM inference container (powered by our friend at [Ollama](https://github.com/ollama/ollama)). With your services up and running, you can use the code chat endpoint as coding-specific LLM chat.
 
-You can now navigate to [`http://localhost:8050/docs`](http://localhost:8050/docs) to interact with the API (or do it through HTTP requests) and explore the documentation.
+### REST API for guideline management & LLM inference
 
-![API Swagger screenshot](docs/quack_api_swagger.png)
+With the service running, you can navigate to [`http://localhost:8050/docs`](http://localhost:8050/docs) to interact with the API (or do it through HTTP requests) and explore the documentation.
 
-In order to stop the service, run:
-```shell
-make stop
-```
+![API Swagger screenshot](docs/quack_api_swagger.png)
 
 ### Latency benchmark
 
@@ -105,8 +100,14 @@ The back-end core feature is to interact with the metadata tables. For the servi
 
 - [Docker](https://docs.docker.com/engine/install/)
 - [Docker compose](https://docs.docker.com/compose/)
+- [GitHub user Personal Access Token](https://github.com/settings/tokens?type=beta) with no extra permissions (= read-only)
+- [Poetry](https://python-poetry.org/docs/) (optional if you run the prod version)
 - [Make](https://www.gnu.org/software/make/) (optional)
 
+If you want to run the LLM on GPU, you'll also need:
+- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+- 1 NVidia GPU (at least 6Gb VRAM recommended for a good performance/latency balance)
+
 The project was designed so that everything runs with Docker orchestration (standalone virtual environment), so you won't need to install any additional libraries.
 
 ### Configuration
@@ -121,8 +122,8 @@ This file will have to hold the following information:
 - `POSTGRES_DB`*: a name for the [PostgreSQL](https://www.postgresql.org/) database that will be created
 - `POSTGRES_USER`*: a login for the PostgreSQL database
 - `POSTGRES_PASSWORD`*: a password for the PostgreSQL database
-- `SUPERADMIN_GH_PAT`: the GitHub token of the initial admin access (Generate a new token on [GitHub](https://github.com/settings/tokens?type=beta), with no extra permissions = read-only)
-- `SUPERADMIN_PWD`*: the password of the initial admin access
+- `SUPERADMIN_GH_PAT`: the GitHub personal access token of the initial admin user
+- `SUPERADMIN_PWD`*: the password of the initial admin user
 - `GH_OAUTH_ID`: the Client ID of the GitHub Oauth app (Create an OAuth app on [GitHub](https://github.com/settings/applications/new), pointing to your Quack dashboard w/ callback URL)
 - `GH_OAUTH_SECRET`: the secret of the GitHub Oauth app (Generate a new client secret on the created OAuth app)
 
@@ -139,6 +140,23 @@ Optionally, the following information can be added:
 - `SUPPORT_EMAIL`: the email used for support of your API.
 - `DEBUG`: if set to false, silence debug logs.
 
+### Running the service
+
+You can start the API containers using this command:
+
+```shell
+make run
+```
+
+You can now navigate to [`http://localhost:8050/docs`](http://localhost:8050/docs) to interact with the API (or do it through HTTP requests) and explore the documentation.
+
+![API Swagger screenshot](docs/quack_api_swagger.png)
+
+In order to stop the service, run:
+```shell
+make stop
+```
+
 
 ## Contributing
 

demo/main.py

@@ -0,0 +1,160 @@
+# Copyright (C) 2024, Quack AI.
+
+# This program is licensed under the Apache License 2.0.
+# See LICENSE or go to <https://www.apache.org/licenses/LICENSE-2.0> for full license details.
+
+import argparse
+import json
+import os
+from pathlib import Path
+from typing import Dict, List
+
+import gradio as gr
+import requests
+from dotenv import load_dotenv
+
+
+class SessionManager:
+    def __init__(self) -> None:
+        self._token = ""
+        self._url = ""
+
+    def set_token(self, token: str) -> None:
+        self._token = token
+
+    def set_url(self, url: str) -> None:
+        self._url = url
+
+    @property
+    def auth(self) -> Dict[str, str]:
+        return {
+            "Authorization": f"Bearer {self._token}",
+            "Content-Type": "application/json",
+        }
+
+    @property
+    def login_endpoint(self) -> str:
+        return f"{self._url}/login/creds"
+
+    @property
+    def chat_endpoint(self) -> str:
+        return f"{self._url}/code/chat"
+
+
+session_manager = SessionManager()
+
+
+def get_token(endpoint: str, login: str, pwd: str) -> str:
+    response = requests.post(
+        endpoint,
+        data={"username": login, "password": pwd},
+        timeout=2,
+    )
+    if response.status_code != 200:
+        raise ValueError(response.json()["detail"])
+    return response.json()["access_token"]
+
+
+def auth_gradio(username: str, password: str) -> bool:
+    try:
+        session_manager.set_token(get_token(session_manager.login_endpoint, username, password))
+        return True
+    except ValueError:
+        return False
+    return False
+
+
+def chat_response(message: str, history: List[List[str]]) -> str:
+    session = requests.Session()
+    _history = [
+        {"role": "user" if idx % 2 == 0 else "assistant", "content": msg}
+        for hist in history
+        for idx, msg in enumerate(hist)
+    ]
+    with session.post(
+        session_manager.chat_endpoint,
+        json={"messages": [*_history, {"role": "user", "content": message}]},
+        headers=session_manager.auth,
+        stream=True,
+    ) as response:
+        reply = ""
+        for line in response.iter_lines():
+            reply += json.loads(line.decode("utf-8"))["message"]["content"]
+            yield reply
+
+
+def main(args: argparse.Namespace) -> None:
+    session_manager.set_url(args.api)
+    # Run the interface
+    folder = Path(__file__).resolve().parent
+    interface = gr.ChatInterface(
+        chat_response,
+        chatbot=gr.Chatbot(
+            elem_id="chatbot",
+            label="Quack Companion",
+            avatar_images=(
+                folder.joinpath("assets", "profile-user.png"),
+                "https://www.quackai.com/_next/image?url=%2Fquack.png&w=64&q=75",
+            ),
+            likeable=True,
+            bubble_full_width=False,
+        ),
+        textbox=gr.Textbox(placeholder="Ask me anything about programming", container=False, scale=7),
+        title="Quack AI: type smarter, ship faster",
+        retry_btn=None,
+        undo_btn=None,
+        css=folder.joinpath("styles", "custom.css"),
+        examples=[
+            # Build
+            "Write a Python function to compute the n-th Fibonacci number",
+            "Write a single-file FastAPI app using SQLModel with a table of users with the fields login, hashed_password and age",
+            "I need a minimal Next JS app to display generated images in a responsive way (mobile included)",
+            "I'm using GitHub Workflow, write the YAML file to lint my src/ folder using ruff",
+            # Fix/improve
+        ],
+        theme=gr.themes.Default(
+            text_size="sm",
+            font=[
+                gr.themes.GoogleFont("Noto Sans"),
+                gr.themes.GoogleFont("Roboto"),
+                "ui-sans-serif",
+                "system-ui",
+                "sans-serif",
+            ],
+            primary_hue="purple",
+            secondary_hue="purple",
+        ),
+        fill_height=True,
+        submit_btn=gr.Button("", variant="primary", size="sm", icon=folder.joinpath("assets", "paper-plane.png")),
+        stop_btn=gr.Button("", variant="stop", size="sm", icon=folder.joinpath("assets", "stop-button.png")),
+    )
+    if not args.auth:
+        session_manager.set_token(
+            get_token(session_manager.login_endpoint, os.environ["SUPERADMIN_LOGIN"], os.environ["SUPERADMIN_PWD"])
+        )
+    interface.launch(
+        server_port=args.port,
+        show_error=True,
+        favicon_path=folder.joinpath("assets", "favicon.ico"),
+        auth=auth_gradio if args.auth else None,
+        show_api=False,
+    )
+
+
+if __name__ == "__main__":
+    load_dotenv()
+    parser = argparse.ArgumentParser(
+        description="Quack API demo",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    parser.add_argument("--port", type=int, default=8001, help="Port on which the webserver will be run")
+    parser.add_argument(
+        "--api",
+        type=str,
+        default=os.getenv("API_URL", "http://localhost:8050/api/v1"),
+        help="URL of your Quack API instance",
+    )
+    parser.add_argument("--auth", action="store_true", help="Use the creds from Auth UI instead of env variables")
+    args = parser.parse_args()
+
+    main(args)

demo/styles/custom.css

@@ -0,0 +1,29 @@
+footer {
+    visibility: hidden
+}
+
+[class *= "message"] {
+    border-radius: var(--radius-xl) !important;
+    border: none;
+}
+.contain { display: flex; flex-direction: column; }
+.gradio-container { height: 100vh !important; }
+#component-0 { height: 100%; }
+#chatbot { flex-grow: 1; overflow: auto; }
+#component-6 { border: none; }
+
+::-webkit-scrollbar {
+    width: 20px;
+}
+::-webkit-scrollbar-track {
+    background-color: transparent;
+}
+::-webkit-scrollbar-thumb {
+    background-color: #adadad;
+    border-radius: 20px;
+    border: 6px solid transparent;
+    background-clip: content-box;
+}
+::-webkit-scrollbar-thumb:hover {
+    background-color: #d6dee1;
+}