Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ytava committed Dec 8, 2022
1 parent 3da7e25 commit d7798f1
Showing 1 changed file with 93 additions and 11 deletions.
104 changes: 93 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,100 @@ HCCL OFI wrapper introduced to act as a thin layer connecting between HCCL and l
# Version
1.0.<br />

# HCCL OFI Wrapper
To use HCCL over libfabric, this wrapper should be built.<br />
# Prerequisites
To use HCCL over OFI libfabric:
1. libfabric should be downloaded and installed.<br />
2. HCCL OFI Wrapper should be downloaded and built.<br />

## Download and install libfabric
libfabric should be downloaded and installed in order to use it.<br />
Please follow the instructions below:<br />
1. Define required version to be installed
```
export REQUIRED_VERSION=1.16.1
```
2. Download libfabric tarball from: https://github.com/ofiwg/libfabric/releases
```
wget https://github.com/ofiwg/libfabric/releases/download/v$REQUIRED_VERSION/libfabric-$REQUIRED_VERSION.tar.bz2 -P /tmp/libfabric
```
3. Store temporary download directory in stack
```
pushd /tmp/libfabric
```
4. Open the file
```
tar -xf libfabric-$REQUIRED_VERSION.tar.bz2
```
5. Define libfabric root location
```
export LIBFABRIC_ROOT=<libFabric library location>
```
6. Create folder for libfabric
```
mkdir -p ${LIBFABRIC_ROOT}
```
7. Change permissions for libfabric folder
```
chmod 777 ${LIBFABRIC_ROOT}
```
8. Change directory to libfabric folder created after opening tar file
```
cd libfabric-$REQUIRED_VERSION/
```
9. Configure libfabric
```
./configure --prefix=$LIBFABRIC_ROOT --enable-debug
```
10. Build and install libfabric
```
make -j 32 && make install
```
11. Remove temporary download directory from stack
```
popd
```
12. Delete temporary download directory
```
rm -rf /tmp/libfabric
```
13. Include LIBFABRIC_ROOT in LD_LIBRARY_PATH:
```
export LD_LIBRARY_PATH=$LIBFABRIC_ROOT/lib:$LD_LIBRARY_PATH
```
Required steps:
1. Download and install libfabric
2. `git clone https://github.com/HabanaAI/hccl_ofi_wrapper.git`
3. `export LIBFABRIC_ROOT=<libFabric library location>`
4. `cd hccl_ofi_wrapper`
5. `make`
6. `cp libhccl_ofi_wrapper.so /usr/lib/habanalabs/libhccl_ofi_wrapper.so`
7. `ldconfig`
8. `export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<hccl_ofi_wrapper.so location>`
Installation can be verified by running: `fi_info --version`.<br />
For more information please see: https://github.com/ofiwg/libfabric
## Build HCCL OFI wrapper
To use libfabric library, HCCL OFI wrapper should be built.<br />
Please follow the instructions below:<br />
1. Clone wrapper from https://github.com/HabanaAI/hccl_ofi_wrapper
```
git clone https://github.com/HabanaAI/hccl_ofi_wrapper.git
```
2. Define LIBFABRIC_ROOT
```
export LIBFABRIC_ROOT=/tmp/libfabric-1.16.0
```
3. Change directory to hccl_ofi_wrapper
```
cd hccl_ofi_wrapper
```
4. Build wrapper
```
make
```
5. Copy wrapper to /usr/lib/habanalabs/
```
cp libhccl_ofi_wrapper.so /usr/lib/habanalabs/libhccl_ofi_wrapper.so
```
6. Run ldconfig utility
```
ldconfig
```
7. Include libhccl_ofi_wrapper.so location in LD_LIBRARY_PATH:
```
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/habanalabs/
```

0 comments on commit d7798f1

Please sign in to comment.