Dockerfile script to build a docker container for oobabooga/text-generation-webui with conda-ready environment

For low GPU VRAM-capable machines

Requirements

1. You have gnu/linux or macOS
2. You have at least 64 GB of RAM
3. You have docker engine installed
4. You have an NVIDIA gpu with >=6GB VRAM and supporting CUDA compute compatibility >=7.5

Preliminary steps

Please, make sure you have fully installed/configure the NVIDIA Container Toolkit checking this guide.

To build the container

chmod +x build start stop rm

./build

To start the container

./start

To connect locally

docker container attach text-generation-webui

To stop the container

./stop

To remove the container

./rm

Note

• Currently, GPTQ-for-LLaMa and the monkey patch are not included