Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Docs] Add MLX backend #251

Merged
merged 2 commits into from
Sep 25, 2024
+23 −0
Merged

[Docs] Add MLX backend #251

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
30c44a0
[Docs] Add MLX backend
grorge123 Sep 9, 2024
4b6c7da
[Docs] fix mac dynamic library name
grorge123 Sep 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions docs/contribute/source/plugin/wasi_nn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,3 +452,26 @@ If the built `wasmedge` CLI tool cannot find the WASI-NN plug-in, you can set th
:::

Then you will have an executable `wasmedge` runtime under `/usr/local/bin` and the WASI-NN with ChatTTS backend plug-in under `/usr/local/lib/wasmedge/libwasmedgePluginWasiNN.so` after installation.


## Build WasmEdge with WASI-NN MLX Backend

You can directly build and install WasmEdge from source or custom install mlx and set `CMAKE_INSTALL_PREFIX` variable.

Build and install WasmEdge from source:
``` bash
cd <path/to/your/wasmedge/source/folder>

cmake -GNinja -Bbuild -DCMAKE_BUILD_TYPE=Release -DWASMEDGE_PLUGIN_WASI_NN_BACKEND="mlx"
cmake --build build

# For the WASI-NN plugin, you should install this project.
cmake --install build
```

<!-- prettier-ignore -->
:::note
If the built `wasmedge` CLI tool cannot find the WASI-NN plug-in, you can set the `WASMEDGE_PLUGIN_PATH` environment variable to the plug-in installation path (such as `/usr/local/lib/wasmedge/`, or the built plug-in path `build/plugins/wasi_nn/`) to try to fix this issue.
:::

Then you will have an executable `wasmedge` runtime under `/usr/local/bin` and the WASI-NN with MLX backend plug-in under `/usr/local/lib/wasmedge/libwasmedgePluginWasiNN.dylib` after installation.
Loading