Skip to content

Developer Guide: Set up MLIR based HeteroCL

Jump to bottom
Hongzheng Chen edited this page Jul 20, 2023 · 11 revisions

This post is a step guide for developers working on zhang research servers about setting up HeteroCL.

Setting up your work environment

SSH

We highly recommend logging in to our research servers with ssh keys instead of passwords. On your laptop, first, generate an ssh key if you haven't done it before:

ssh-keygen

Press return all the way, then you should find a public key here: ~/.ssh/id_rsa. Next, add the following lines to your ~/.ssh/config file. If you don't have this file, create it.

Host zhang-21
    HostName zhang-21.ece.cornell.edu
    User your_netid

Host zhang-x1
    HostName zhang-x1.ece.cornell.edu
    User your_netid

Lastly, do ssh-copy-id zhang-21 and ssh-copy-id zhang-x1 and type your password. After that, you should be able to log in to the servers without the password, by ssh zhang-21, or ssh zhang-x1.

Conda

It is recommended to use Python virtual environment. If you haven't done so, install conda: https://docs.anaconda.com/anaconda/install/linux/

Note that this step happens on the server.

Create an environment, feel free to change the name. Python version >= 3.8 is recommended.

conda create -n mlir python=3.8

Optionally, you can set up activate.d and deactivate.d script to set certain env vars when you do conda activate your_env. Check out this stackoverflow answer to see how: https://stackoverflow.com/questions/31598963/how-to-set-specific-environment-variables-when-activating-conda-environment. If you don't worry about managing shared env vars, you can just set env vars in your ~/.bashrc.

Make tools and gcc

We need a newer version of gcc to build our project. Add this line to your ~/.bashrc:

source /opt/rh/devtoolset-7/enable

CMake

The default cmake is too old, we use a newer version to build our dialect. Add this line to your ~/.bashrc:

# CMake
export PATH=/work/shared/common/cmake-3.23.3/bin/:$PATH

Verify that it works by cmake --version, you should see:

$ cmake --version
cmake version 3.23.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

LLVM

We have a shared LLVM project, you don't have to build your own.

Add these lines to your ~/.bashrc

# LLVM
export LLVM_HOME=/work/shared/common/llvm-project-hcl
export PREFIX=$LLVM_HOME
export LLVM_BUILD_DIR=$LLVM_HOME/build
export LLVM_SYMBOLIZER_PATH=$LLVM_HOME/build/bin/llvm-symbolizer
export PATH=$LLVM_BUILD_DIR/bin:$PATH
# LLVM Python Bindings
export PYTHONPATH=$LLVM_HOME/build/tools/mlir/python_packages/mlir_core:$PYTHONPATH

Go to LLVM directory and install its Python dependencies:

cd /work/shared/common/llvm-project-hcl
python3 -m pip install -r mlir/python/requirements.txt

Verify that LLVM is working:

$ mlir-opt --version 
LLVM (http://llvm.org/):
  LLVM version 15.0.0
  Optimized build with assertions.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: cascadelake

You should be able to import mlir package like this:

$ python
Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:59:51) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mlir
>>>

Build HCL Dialect

Do not build the dialect in your home directory. The home directory has limited disk space quota, and it should only hold your configuration files. Instead, clone HeteroCL and HCL dialect in your shared directory. Your shared directory is /work/shared/users/ugrad or meng/your_netid.

In your shared directory:

git clone https://github.com/cornell-zhang/hcl-dialect.git
cd hcl-dialect
mkdir build && cd build
cmake -G "Unix Makefiles" .. \
   -DMLIR_DIR=$LLVM_BUILD_DIR/lib/cmake/mlir \
   -DLLVM_EXTERNAL_LIT=$LLVM_BUILD_DIR/bin/llvm-lit \
   -DPYTHON_BINDING=ON \
   -DOPENSCOP=OFF \
   -DPython3_EXECUTABLE=`which python`
make -j

Next, add following env vars to your ~/.bashrc, or your activate.d if you prefer to enable them only when you use this conda environment. You need to change the path to your work directory

# HeteroCL Dialect Python Binding
export HCL_DIALECT_HOME=/work/shared/users/path/to/hcl-dialect # you need to edit this
export HCL_DIALECT_BUILD_DIR=$HCL_DIALECT_HOME/build/
export PYTHONPATH=$HCL_DIALECT_BUILD_DIR/tools/hcl/python_packages/hcl_core:$PYTHONPATH

# HeteroCL Dialect Commandline Interface
export PATH=$HCL_DIALECT_BUILD_DIR/bin:$PATH

# HeteroCL Runtime Library
export LD_LIBRARY_PATH=$HCL_DIALECT_BUILD_DIR/lib:$LD_LIBRARY_PATH

To see if the dialect works:

cd /your/path/to/hcl-dialect/build
cmake --build . --target check-hcl

This command runs the unit tests with hcl dialect. If all passed, you should see a green "success" text prompt.

You should be able to do this:

$ python
Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:59:51) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hcl_mlir
>>>

Set up HeteroCL frontend

In your shared directory, clone HeteroCL and switch to hcl-mlir branch

git clone https://github.com/cornell-zhang/heterocl.git
cd heterocl
git checkout hcl-mlir

No need to build it because it's all Python code. Install dependencies:

python -m pip install -r python/requirements.txt

Add these lines to your ~/.bashrc or conda activate.d.

# HeteroCL Frontend Python Package
export HCL_HOME=/work/shared/users/path/to/heterocl # you need to edit this path
export PYTHONPATH=$HCL_HOME/python:$PYTHONPATH

Verify the HeteroCL works end-to-end:

cd /your/path/to/heterocl
python -m pytest tests -v

Troubleshooting

CLIB error

If you see errors like this during IR system's compilation, switch to zhang-21. This is caused by mismatching library versions.

/work/shared/common/llvm-project-hcl/build/bin/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /work/shared/common/llvm-project-hcl/build/bin/mlir-tblgen)
/work/shared/common/llvm-project-hcl/build/bin/mlir-tblgen: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /work/shared/common/llvm-project-hcl/build/bin/mlir-tblgen)
/work/shared/common/llvm-project-hcl/build/bin/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /work/shared/common/llvm-project-hcl/build/bin/mlir-tblgen)
Clone this wiki locally