[Web] WebAssembly memory is not detachable for Web Worker

[Honry]

Describe the issue

Issue introduced in 07cfc56#diff-66aa0acafe57e6847abcda96de26b731b3e32e44cd825f3a50c6afbe5ff3985eR52, for model with size > 1G, it uses WebAssembly Memory to allocate larger ArrayBuffer, such memory is not detachable and could not be transferred to Worker thread. Error will be threw from https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/wasm/proxy-wrapper.ts#L140.

To reproduce

• Enable ort.env.wasm.proxy to true,
• Create session with a large model path which size is > 1G.

Error:

TypeError: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 0 is not detachable and could not be transferred.

Urgency

No response

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

07cfc56

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

[Honry]

cc / @fs-eire

[fs-eire]

Let me check if I can fix this.

[fs-eire]

@Honry I made this fix which should allow proxy loading larger models. However, I am thinking of deprecating proxy worker support in future.

The feature will not be removed. It will be there, but will only support CPU and no longer to support new features.

Some features (eg. IO-Binding) are difficult to implement using proxy worker. For example, in WebGPU, you cannot create a GPUBuffer in main thread because it requires a device object and that object will be in the worker's context. I am not familiar with WebNN but I assume that similar limitations will occur.

ort-web already support being loaded in a worker via importScripts(). I didn't test that for WebNN but I think it should already work (or maybe need a small fix on disabling enforcing proxy is ON). Being able to be loaded in worker makes ort-web the same capability to support proxy worker and it's going to be more efficient.

[Honry]

@fs-eire, thanks for your detailed explanation, that make sense, I will try to enable WebNN async API firstly and hope there's no performance regression.