Skip to content

Latest commit

 

History

History
49 lines (33 loc) · 1.19 KB

README.md

File metadata and controls

49 lines (33 loc) · 1.19 KB
Raw

CarpAI-LLM

This is a docker image to test the implementation of an LLM on the lilypad stack.

Requierements

  • docker

  • docker compose

  • hugging face key

Usage

The script takes one argument, the CID of the ipfs JSON file to be used as input

usage: main.py [-h] --i IPFS

Lilypad LLM

options:
  -h, --help  show this help message and exit
  --i IPFS    IPFS Hash

JSON Inputs

The script uses IPFS files as inputs to do inferences, The JSON files must have the following formating, which follows langchain formats:

{
    "template": "You are a friendly chatbot assistant that responds conversationally to users' questions. \n Keep the answers short, unless specifically asked by the user to elaborate on something. \n \n Question: {question} \n \n Answer:",
    "parameters": {"question": "What is a chatbot?"}
}

Docker

  1. build the image using the docker compose stack:
  • docker compose build
  1. Upload the image to the docker hub
  • docker tag CID HUB/NAME:VERSION
  1. Use the image with bacalhau
  • bacalhau docker run --gpu 1 HUB/NAME:VERSION -- python main.py --i CID
  1. Use this image with lilypad /!\ DOES NOT WORK YET /!\
  • lilypad run fastchat:v0.0.1 CID