Install Sparse Attention on a Vast.ai Instance

[afiaka87]

Requirements:

• vast.ai account
• V100 (nothing else is really supported)
• Instance is running container: ufoym/all-jupyter-10.2

• 50 GiB hard drive space to be safe.

#!/bin/bash

# update the apt software source index
apt-get -y update
apt-get -y install git
apt-get -y install clang-10
apt-get -y install llvm-9-dev
apt-get -y install cmake
apt-get -y install wget
apt-get -y install software-properties-common
apt-get -y install yes


wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
eval "$(/root/miniconda3/bin/conda shell.bash hook)"
conda init
conda create -n tr python=3.6
conda activate tr
conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
pip install triton

git clone https://github.com/microsoft/DeepSpeed.git /tmp/Deepspeed
cd /tmp/Deepspeed && DS_BUILD_SPARSE_ATTN=1 ./install.sh -s # You will need to manually change this to -r if you're root