README.md

Groq

Connection to Groq

Login to the Groq login node from your local machine. Once you are on the login node, ssh to one of the Groq nodes.

local > ssh [email protected]

groq-login > ssh groq-r01-gn-01.ai.alcf.anl.gov
# or
groq-login > ssh groq-r01-gn-09.ai.alcf.anl.gov
# or any node with hostname of form groq-r01-gn-0[1-9].ai.alcf.anl.gov

Prerequisite: Create Virtual Environment

Install Miniconda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

PyTorch virtual environment

export PYTHON_VERSION=3.10.12
conda create -n groqflow python=$PYTHON_VERSION
conda activate groqflow

Install Groqflow

git clone https://github.com/groq/groqflow.git
cd groqflow
pip install --upgrade pip
pip install -e .
pushd . 
cd demo_helpers
pip install -e .
popd

Job Queuing and Submission

Groq jobs in the AI Testbed's Groqrack are managed by the PBS job scheduler.

• qsub : to submit a batch job using a script
• qstat: to display queue information
• qdel: to delete (cancel) a job:
• qhold: to hold a job

Schedule Interactive Job

Following command gives a single Groq node interactively for 1 hour

qsub -I -l walltime=1:00:00 

Other flags that can be used

-l ncpus=1 
-l groq_accelerator=1

Hands-on Example

• Bert

Homework

Run BERT example with custom input instead of dummy input.

Additional Examples (Optional)

• ResNet50
• MiniLMv2

Useful Resources

• ALCF Groq Documenation
• Groq Documentation
• Groq Examples Repository