Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dml build instructions for onnxruntime-genai #20179

Closed
wants to merge 7 commits into from
Closed

Add dml build instructions for onnxruntime-genai #20179

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8f94803
Add DirectML build from source instructions
natke Apr 2, 2024
117c877
Fix typo
natke Apr 2, 2024
af92743
Merge branch 'gh-pages' into add-dml-build-instructions
natke Apr 2, 2024
7024bb6
Formatting
natke Apr 2, 2024
cc137b9
More formatting
natke Apr 2, 2024
31a634f
More formatting
natke Apr 2, 2024
405d75a
Add --ort_home to the build instructions
natke Apr 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 104 additions & 70 deletions docs/genai/howto/build-from-source.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,105 +17,139 @@ nav_order: 2

`cmake`

## Build steps
## Clone the onnxruntime-genai repo

1. Clone this repo
```bash
git clone https://github.com/microsoft/onnxruntime-genai
cd onnxruntime-genai
```

```bash
git clone https://github.com/microsoft/onnxruntime-genai
cd onnxruntime-genai
```
## Install ONNX Runtime

2. Install ONNX Runtime
By default, the onnxruntime-genai build expects to find the ONNX Runtime include and binaries in a folder called `ort` in the root directory of onnxruntime-genai. You can put the ONNX Runtime files in a different location and specify this location to the onnxruntime-genai build. These instructions use `ORT_HOME` as the location.

By default, the onnxruntime-genai build expects to find the ONNX Runtime include and binaries in a folder called `ort` in the root directory of onnxruntime-genai. You can put the ONNX Runtime files in a different location and specify this location to the onnxruntime-genai build. These instructions use ORT_HOME as the location.
### Option 1: Install from release

* Install from release
These instructions are for the Linux GPU build of ONNX Runtime. Replace `linux-gpu` with your target of choice.

These instructions are for the Linux GPU build of ONNX Runtime. Replace the location with the operating system and target of choice.
```bash
cd <ORT_HOME>
curl -L https://github.com/microsoft/onnxruntime/releases/download/v1.17.0/onnxruntime-linux-x64-gpu-1.17.1.tgz
tar xvzf onnxruntime-linux-x64-gpu-1.17.1.tgz
mv onnxruntime-linux-x64-gpu-1.17.1/include .
mv onnxruntime-linux-x64-gpu-1.17.1/lib .
```

```bash
cd $ORT_HOME
wget https://github.com/microsoft/onnxruntime/releases/download/v1.17.0/onnxruntime-linux-x64-gpu-1.17.0.tgz
tar xvzf onnxruntime-linux-x64-gpu-1.17.0.tgz
mv onnxruntime-linux-x64-gpu-1.17.0/include .
mv onnxruntime-linux-x64-gpu-1.17.0/lib .
```
### Option 2: Install from nightly

* Install from nightly

Download the nightly nuget package `Microsoft.ML.OnnxRuntime` from: https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly.
Download the nightly nuget package `Microsoft.ML.OnnxRuntime` from: https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly.

Extract the nuget package.
Extract the nuget package.

```bash
tar xvf Microsoft.ML.OnnxRuntime.1.18.0-dev-20240322-0323-ca825cb6e6.nupkg
```
```bash
tar xvf Microsoft.ML.OnnxRuntime.1.18.0-dev-20240322-0323-ca825cb6e6.nupkg
```

Copy the include and lib files into $ORT_HOME.
Copy the include and lib files into `ORT_HOME`.

On Windows
On Windows

Example is given for `win-x64`. Change this to your architecture if different.
Example is given for `win-x64`. Change this to your architecture if different.

```cmd
copy build\native\include\onnxruntime_c_api.h $ORT_HOME\include
copy runtimes\win-x64\native\*.dll $ORT_HOME\lib
```
```cmd
copy build\native\include\onnxruntime_c_api.h <ORT_HOME>\include
copy runtimes\win-x64\native\*.dll <ORT_HOME>\lib
```

On Linux
On Linux

```cmd
cp build/native/include/onnxruntime_c_api.h $ORT_HOME/include
cp build/linux-x64/native/libonnxruntime*.so* $ORT_HOME/lib
```
```cmd
cp build/native/include/onnxruntime_c_api.h <ORT_HOME>/include
cp build/linux-x64/native/libonnxruntime*.so* <ORT_HOME>/lib
```

* Or build from source
### Option 3: Build from source

```
git clone https://github.com/microsoft/onnxruntime.git
cd onnxruntime
```

Create include and lib folders in the `ORT_HOME` directory

```bash
mkdir <ORT HOME>/include
mkdir <ORT_HOME>/lib
```

Build from source and copy the include and libraries into `ORT_HOME`

```
git clone https://github.com/microsoft/onnxruntime.git
cd onnxruntime
```
On Windows

Create include and lib folders in the ORT_HOME directory
```cmd
build.bat --build_shared_lib --skip_tests --parallel [--use_cuda]
copy include\onnxruntime\core\session\onnxruntime_c_api.h <ORT_HOME>\include
copy build\Windows\Debug\Debug\*.dll <ORT_HOME>\lib
```

```bash
mkdir $ORT_HOME/include
mkdir $ORT_HOME/lib
```
On Linux

Build from source and copy the include and libraries into ORT_HOME
```cmd
./build.sh --build_shared_lib --skip_tests --parallel [--use_cuda]
cp include/onnxruntime/core/session/onnxruntime_c_api.h <ORT_HOME>/include
cp build/Linux/RelWithDebInfo/libonnxruntime*.so* <ORT_HOME>/lib
```

On Windows
## Build onnxruntime-genai

```cmd
build.bat --build_shared_lib --skip_tests --parallel [--use_cuda]
copy include\onnxruntime\core\session\onnxruntime_c_api.h $ORT_HOME\include
copy build\Windows\Debug\Debug\*.dll $ORT_HOME\lib
```
### Build for CPU

On Linux
```bash
cd ..
python build.py [--ort_home <ORT_HOME>]
```

```cmd
./build.sh --build_shared_lib --skip_tests --parallel [--use_cuda]
cp include/onnxruntime/core/session/onnxruntime_c_api.h $ORT_HOME/include
cp build/Linux/RelWithDebInfo/libonnxruntime*.so* $ORT_HOME/lib
```
### Build for CUDA

3. Build onnxruntime-genai
These instructions assume you already have CUDA installed.

If you are building for CUDA, add the cuda_home argument.
```bash
cd ..
python build.py --cuda_home <path to cuda home> [--ort_home <ORT_HOME>]
```

```bash
cd ..
python build.py [--cuda_home <path_to_cuda_home>]
```
### Build for DirectML

Two extra files are required for the DirectML build of onnxruntime-genai `dml_provider_factory.h` and `DirectML.dll`.

```cmd
cd <ORT_HOME>
curl -L https://github.com/microsoft/onnxruntime/releases/download/v1.17.1/Microsoft.ML.OnnxRuntime.DirectML.1.17.1.zip > Microsoft.ML.OnnxRuntime.DirectML.1.17.1.zip
mkdir Microsoft.ML.OnnxRuntime.DirectML.1.17.1
tar xvf Microsoft.ML.OnnxRuntime.DirectML.1.17.1.zip -C Microsoft.ML.OnnxRuntime.DirectML.1.17.1
copy Microsoft.ML.OnnxRuntime.DirectML.1.17.1\build\native\include\dml_provider_factory.h include
curl -L https://www.nuget.org/api/v2/package/Microsoft.AI.DirectML/1.13.1 > Microsoft.AI.DirectML.1.13.1.nupkg
mkdir Microsoft.AI.DirectML.1.13.1
tar xvf Microsoft.AI.DirectML.1.13.1.nupkg -C Microsoft.AI.DirectML.1.13.1
copy Microsoft.AI.DirectML.1.13.1\bin\x64-win\DirectML.dll lib
```

After the extra files have been copied into <ORT HOME>, build onnxruntime-genai as follows:

```bash
python build.py --use_dml [--ort_home <ORT_HOME>]
```


4. Install Python wheel
### Install the library into your application

#### Install Python wheel

```bash
cd build/wheel
pip install *.whl
```

#### Install Nuget package

```bash
cd build/wheel
pip install *.whl
```
#### Install C/C++ header file and library
Loading