README.md

LeapfrogAI llama-cpp-python Backend

A LeapfrogAI API-compatible instructor-xl model for creating embeddings across CPU and GPU infrastructures.

Usage

Zarf Package Deployment

To build and deploy just the text-embeddings Zarf package (from the root of the repository):

Deploy a UDS cluster if one isn't deployed already

pip install 'huggingface_hub[cli,hf_transfer]'  # Used to download the model weights from huggingface
make build-text-embeddings LOCAL_VERSION=dev
uds zarf package deploy packages/text-embeddings/zarf-package-text-embeddings-*-dev.tar.zst --confirm

Local Development

To run the text-embeddings backend locally (starting from the root directory of the repository):

# Setup Virtual Environment if you haven't done so already
python -m venv .venv
source .venv/bin/activate

# install dependencies
python -m pip install src/leapfrogai_sdk
cd packages/text-embeddings
python -m pip install ".[dev]"

# download the model
python scripts/model_download.py

# start the model backend
python -u main.py