A proposed WebAssembly System Interface API.
Phase 2
- Mendy Berger
- Sean Isom
Linux, Window, MacOS, Android, Web.
- Introduction
- Goals [or Motivating Use Cases, or Scenarios]
- Non-goals
- API walk-through
- Detailed design discussion
- Considered alternatives
- Stakeholder Interest & Feedback
- References & acknowledgements
This proposal is a collection of multiple packages.
| Package | Description |
|---|---|
wasi:webgpu |
WebGPU package for interacting with GPUs |
wasi:frame-buffer |
A CPU based graphics API |
wasi:surface |
A surface |
wasi:graphics-context |
A small interface connecting the graphics APIs (webgpu or frame-buffer) to surface |
- Bring some of the benefits that Wasm provides to CPU code, over to GPU code.
- Enable secure GPU compute.
- Enable wasm-based UI applications.
- Enable wasm-based graphics even in environments with no GPUs.
- Enable wasm-based AI for cases not covered by wasi-nn.
- A full windowing API.
- VR/AR (subject to change, if we find contributors with expertise in the field).
The full API documentation can be found in imports.md.
wasi:webgpu is based on the official webgpu spec.
wasi:webgpu does deviate sometimes from the webgpu spec, namely, in cases where the spec makes assumptions about running in a web or JS environment. Wherever wasi:webgpu deviates from the spec, a clear explanation should be documented.
wasi:surface is a basic surface API. It lets you create a surface you can draw to (similar to canvas on web). You can also get basic user events from the surface, like pointer and keyboard events.
wasi:surface supports the following event types:
- Pointer: For mouse/touch/pencil/etc.
- Key: For keyboard events.
- Resize: Surface was resized.
- Frame: Fires on each frame. (similar to
requestAnimationFrameon the web)
Other user input, such as scroll wheel, clipboard, drag-over, device motion, device orientation, gamepad input, etc., are all useful in many applications, but since they're a bit less common, they're out of scope for now. However, they might be added eventually.
A simple frame buffer API for CPU-based rendering.
This might be especially useful in embedded.
wasi:graphics-context is the point of connection between a graphics API (wasi:webgpu, wasi:frame-buffer, future graphics API) to a windowing system (wasi:surface, future windowing system). See diagram.
wasi:graphics-context has a method to get the next frame as an abstract-buffer. Abstract buffer's contents can't be read. Instead, the graphics API in use can turn the abstract-buffer into something it can make use of. E.g. webgpu is able to turn the buffer into a GPUTexture. The runtime can easily, in the background, represent the abstract-buffer as a GPUTexture, and the conversion would just be a no-op.
[Provide example code snippets and diagrams explaining how the API would be used to solve the given problem]
[etc.]
[This section should mostly refer to the .wit.md file that specifies the API. This section is for any discussion of the choices made in the API which don't make sense to document in the spec file itself.]
[Talk through the tradeoffs in coming to the specific design point you want to make.]
// Illustrated with example code.
[This may be an open question, in which case you should link to any active discussion threads.]
[etc.]
[This section is not required if you already covered considered alternatives in the design discussion above.]
[Describe an alternative which was considered, and why you decided against it.]
[etc.]
TODO before entering Phase 3.
[This should include a list of implementers who have expressed interest in implementing the proposal]
Many thanks for valuable feedback and advice from:
- [Person 1]
- [Person 2]
- [etc.]