Support loading models by name #38

geekbeast · 2023-04-03T01:32:02Z

This change has been scoped down to only adding support for getting a handle to a named model for inference and updating the WIT bindings to the latest spec.

Also added in typenames to make wit-bindgen generation more straightforward.

wasi-nn.witx

mingqiusun · 2023-04-14T18:44:29Z

I think the register/deregister methods make sense for the FaaS scenario, but optional/unnecessary for a standalone env. We should document those two use cases, maybe via sample programs?

…ead of resources

pchickey · 2023-08-07T19:36:55Z

wit/wasi-nn.wit

+    import execution
+    import errors
+
+    export run: func()-> result<_,string>


A world definition that exports a run like this is meant for a program that imports those four interfaces, takes no arguments, and has no other imports. I don't think that is actually a world anyone wants to target.

My expectation is that downstream of this spec, embedders will provide the wasi-nn imports, but the export funcs will be part of some other spec. e.g. if its a CLI program the export will be provided by the wasi:cli/command world, and if its used in a web server the export will be provided by the wasi:http/proxy world.

This may have been a stray experimentation error while I was trying to track down what was causing the wit-bindgen failure. This can be removed entirely.

That said, I wasn't 100% clear on what is going to get exported here for use by other components. Do the graph and execution interfaces need to be exported so that other components can perform inference?

a world is from the perspective of the user, as opposed to an implementer: import in the world definition says, if I'm writing a component that uses inference I have these interfaces available to import. the implementer of a world needs to make imports available as exports.

wit/wasi-nn.wit

wasi-nn.witx

wit/wasi-nn.wit

pchickey · 2023-08-07T19:44:09Z

README.md

@@ -110,7 +110,7 @@ used to solve the given problem.

 ### Detailed design discussion

-For the details of the API, see [wasi-nn.wit.md](wasi-nn.wit.md).
+For the details of the API, see [wasi-nn.wit.md](legacy_wit/wasi-nn.wit.md).


unclear to me why this file was renamed and why the pointer is to it instead of to wit/wasi-nn.wit

I will just delete it and point it at the new file. If someone needs access they can pull from git history.

wasi-nn.witx

wit/wasi-nn.wit

This change refactors the `wasmtime-wasi-nn` crate to allow access from both `preview1` and `preview2` ABIs. Though the `wasi-nn` specification has included a WIT description for some time, here we use some in-tree files until WebAssembly/wasi-nn#38 is landed. The `preview2` code is not exercised anywhere yet: ideally this would be wired up once component model `resource`s are fully implemented in Wasmtime.

This change refactors the `wasmtime-wasi-nn` crate to allow access from both `preview1` and `preview2` ABIs. Though the `wasi-nn` specification has included a WIT description for some time, here we use some in-tree files until WebAssembly/wasi-nn#38 is landed. The `preview2` code is not exercised anywhere yet: ideally this would be wired up once component model `resource`s are fully implemented in Wasmtime. prtest:full

abrown

Ok, let's merge this. This is enough to implement "named models" and we can iterate on these definitions as we go along. (And I guess I'll fix the failing CI in another PR).

This change also removes some extra functions that snuck through the review of WebAssembly#38: `set-input-by-name`, `eval`.

* Move original documentation to new `wasi-nn.wit` file This change also removes some extra functions that snuck through the review of #38: `set-input-by-name`, `eval`. * Move `wasi-nn.wit` to top-level directory * Remove original `*.wit.md` documents * Tweak `get-output` return type

@pchickey

* wasi-nn: refactor to allow `preview2` access This change refactors the `wasmtime-wasi-nn` crate to allow access from both `preview1` and `preview2` ABIs. Though the `wasi-nn` specification has included a WIT description for some time, here we use some in-tree files until WebAssembly/wasi-nn#38 is landed. The `preview2` code is not exercised anywhere yet: ideally this would be wired up once component model `resource`s are fully implemented in Wasmtime. prtest:full * wasi-nn: use `preview1` linkage prtest:full * review: rename `preview*` to `wit*` This is based on @pchickey's [comments] on ABI naming. [comments]: https://bytecodealliance.zulipchat.com/#narrow/stream/266558-wasi-nn/topic/wasi-nn.20.2B.20preview2/near/383368292 * review: update README * fix: remove broken doc links * fix: replace typo `wit` with `gen` * review: use `wit` types everywhere This removes the crate-specific types in order to use the WIT-generated types throughout the crate. The main effect of this is that the crate no longer optionally includes `wasmtime` with the `component-model` feature--now that is required. * review: move `BackendKind` conversion into `witx.rs` * review: remove `<'a>` * review: use `tracing` crate instead of `eprintln!`

@pchickey

* wasi-nn: refactor to allow `preview2` access This change refactors the `wasmtime-wasi-nn` crate to allow access from both `preview1` and `preview2` ABIs. Though the `wasi-nn` specification has included a WIT description for some time, here we use some in-tree files until WebAssembly/wasi-nn#38 is landed. The `preview2` code is not exercised anywhere yet: ideally this would be wired up once component model `resource`s are fully implemented in Wasmtime. prtest:full * wasi-nn: use `preview1` linkage prtest:full * review: rename `preview*` to `wit*` This is based on @pchickey's [comments] on ABI naming. [comments]: https://bytecodealliance.zulipchat.com/#narrow/stream/266558-wasi-nn/topic/wasi-nn.20.2B.20preview2/near/383368292 * review: update README * fix: remove broken doc links * fix: replace typo `wit` with `gen` * review: use `wit` types everywhere This removes the crate-specific types in order to use the WIT-generated types throughout the crate. The main effect of this is that the crate no longer optionally includes `wasmtime` with the `component-model` feature--now that is required. * review: move `BackendKind` conversion into `witx.rs` * review: remove `<'a>` * review: use `tracing` crate instead of `eprintln!`

geekbeast added 4 commits April 1, 2023 14:37

Add registry and named load to witx

0a7b548

Add functions for managing registation.

b20e964

Also added in typenames to make wit-bindgen generation more straightforward.

Fix errant case of not using utfname type.

f7dc344

Add ttl and remove uri based function.

5fa53e3

abrown reviewed Apr 12, 2023

View reviewed changes

wasi-nn.witx Outdated Show resolved Hide resolved

wasi-nn.witx Outdated Show resolved Hide resolved

wasi-nn.witx Outdated Show resolved Hide resolved

wasi-nn.witx Outdated Show resolved Hide resolved

geekbeast added 2 commits April 11, 2023 15:16

Change function name to match proposal.

2648c6e

Use string and remove ttl.

736462f

geekbeast commented Apr 12, 2023

View reviewed changes

wasi-nn.witx Outdated Show resolved Hide resolved

wasi-nn.witx Outdated Show resolved Hide resolved

wasi-nn.witx Outdated Show resolved Hide resolved

abrown mentioned this pull request Apr 13, 2023

feature: add "named models" #36

Closed

Matthew Tamayo-Rios added 4 commits April 19, 2023 15:16

Remove is_registered and add list models.

62b482c

Fix types.

8b516f7

Update for returning model list.

d54aa2a

Update WIT and clean up WITX.

31d6207

geekbeast requested a review from pchickey August 7, 2023 19:02

geekbeast changed the title ~~Feature/named models~~ Support loading models by name Aug 7, 2023

geekbeast requested a review from abrown August 7, 2023 19:16

Matthew Tamayo added 2 commits August 7, 2023 12:20

Uncomment graph and execution after switching to u32 for handles inst…

5591ec6

…ead of resources

Add additional error types.

d6811b0

pchickey reviewed Aug 7, 2023

View reviewed changes

Matthew Tamayo added 2 commits August 7, 2023 21:17

Clean up enums and fix other PR feedback.

66f91ff

Add set input by name.

208750d

geekbeast requested a review from pchickey August 8, 2023 07:04

abrown mentioned this pull request Aug 8, 2023

Fix the error types #42

Open

abrown reviewed Aug 8, 2023

View reviewed changes

wasi-nn.witx Outdated Show resolved Hide resolved

wasi-nn.witx Outdated Show resolved Hide resolved

wit/wasi-nn.wit Show resolved Hide resolved

wit/wasi-nn.wit Show resolved Hide resolved

wit/wasi-nn.wit Show resolved Hide resolved

wit/wasi-nn.wit Show resolved Hide resolved

abrown mentioned this pull request Aug 8, 2023

wasi-nn: refactor to allow preview2 access bytecodealliance/wasmtime#6821

Merged

Fix PR feedback.

53a2e11

abrown approved these changes Aug 9, 2023

View reviewed changes

abrown merged commit c1ff124 into WebAssembly:main Aug 9, 2023
1 check failed

abrown added a commit to abrown/wasi-nn-spec that referenced this pull request Aug 9, 2023

ci: fix wit-abi error introduced in WebAssembly#38

781fc8d

abrown added a commit that referenced this pull request Aug 9, 2023

ci: fix wit-abi error introduced in #38 (#44)

346bf5d

CaptainVincent mentioned this pull request Aug 11, 2023

Bump version to v0.5.0 bytecodealliance/wasi-nn#87

Merged

abrown added a commit to abrown/wasi-nn-spec that referenced this pull request Aug 14, 2023

Move original documentation to new wasi-nn.wit file

e6a77c9

This change also removes some extra functions that snuck through the review of WebAssembly#38: `set-input-by-name`, `eval`.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support loading models by name #38

Support loading models by name #38

geekbeast commented Apr 3, 2023 •

edited

Loading

mingqiusun commented Apr 14, 2023

pchickey Aug 7, 2023

geekbeast Aug 8, 2023

pchickey Aug 8, 2023

pchickey Aug 7, 2023

geekbeast Aug 8, 2023

abrown left a comment •

edited

Loading

Support loading models by name #38

Support loading models by name #38

Conversation

geekbeast commented Apr 3, 2023 • edited Loading

mingqiusun commented Apr 14, 2023

pchickey Aug 7, 2023

Choose a reason for hiding this comment

geekbeast Aug 8, 2023

Choose a reason for hiding this comment

pchickey Aug 8, 2023

Choose a reason for hiding this comment

pchickey Aug 7, 2023

Choose a reason for hiding this comment

geekbeast Aug 8, 2023

Choose a reason for hiding this comment

abrown left a comment • edited Loading

Choose a reason for hiding this comment

geekbeast commented Apr 3, 2023 •

edited

Loading

abrown left a comment •

edited

Loading