Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[js/webnn] update API of session options for WebNN #20816

Merged
merged 3 commits into from
May 31, 2024

Conversation

fs-eire
Copy link
Contributor

@fs-eire fs-eire commented May 24, 2024

Description

This PR is an API-only change to address the requirements being discussed in #20729.

There are multiple ways that users may create an ORT session by specifying the session options differently.

All the code snippet below will use the variable webnnOptions as this:

const myWebnnSession = await ort.InferenceSession.create('./model.onnx', {
   executionProviders: [
     webnnOptions
   ]
});

The old way (backward-compatibility)

// all-default, name only
const webnnOptions_0 = 'webnn';

// all-default, properties omitted
const webnnOptions_1 = { name: 'webnn' };

// partial
const webnnOptions_2 = {
  name: 'webnn',
  deviceType: 'cpu'
};

// full
const webnnOptions_3 = {
  name: 'webnn',
  deviceType: 'gpu',
  numThreads: 1,
  powerPreference: 'high-performance'
};

The new way (specify with MLContext)

// options to create MLcontext
const options = {
  deviceType: 'gpu',
  powerPreference: 'high-performance'
};

const myMlContext = await navigator.ml.createContext(options);

// options for session options
const webnnOptions = {
  name: 'webnn',
  context: myMlContext,
  ...options
};

This should throw (because no deviceType is specified):

const myMlContext = await navigator.ml.createContext({ ... });
const webnnOptions = {
  name: 'webnn',
  context: myMlContext
};

Interop with WebGPU

// get WebGPU device
const adaptor = await navigator.gpu.requestAdapter({ ... });
const device = await adaptor.requestDevice({ ... });

// set WebGPU adaptor and device
ort.env.webgpu.adaptor = adaptor;
ort.env.webgpu.device = device;

const myMlContext = await navigator.ml.createContext(device);
const webnnOptions = {
  name: 'webnn',
  context: myMlContext,
  gpuDevice: device
};

This should throw (because cannot specify both gpu device and MLContext option at the same time):

const webnnOptions = {
  name: 'webnn',
  context: myMlContext,
  gpuDevice: device,
  deviceType: 'gpu'
};

@fs-eire
Copy link
Contributor Author

fs-eire commented May 24, 2024

@Honry @huningxin @egalli Please take a look and let me know if you have any concerns.

Copy link

@huningxin huningxin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with questions, thanks much!

js/web/lib/wasm/session-options.ts Show resolved Hide resolved
js/common/lib/inference-session.ts Show resolved Hide resolved
Copy link
Contributor

@Honry Honry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, this makes things very clear and fulfills all the requirements.

Thanks much @fs-eire!

Copy link
Contributor

@egalli egalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for making all the API changes!

@guschmue guschmue added the ep:WebNN WebNN execution provider label May 29, 2024
@fs-eire fs-eire merged commit 35697d2 into main May 31, 2024
96 checks passed
@fs-eire fs-eire deleted the fs-eire/webnn-mlcontext-api branch May 31, 2024 10:25
egalli added a commit to egalli/onnxruntime that referenced this pull request Jun 10, 2024
This change enables the API added in microsoft#20816 as well as moving context creation
to JS.
guschmue pushed a commit that referenced this pull request Jul 8, 2024
### Description
This PR enables the API added in #20816 as well as moving context
creation to JS.

### Motivation and Context
In order to enable I/O Binding with the upcoming
[MLBuffer](webmachinelearning/webnn#542) API
in the WebNN specification, we need to share the same `MLContext` across
multiple sessions. This is because `MLBuffer`s are restricted to the
`MLContext` where they were created. This PR enables developers to use
the same `MLContext` across multiple sessions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ep:WebNN WebNN execution provider
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants