[Feature request] Deno webgpu support #22309

jlucaso1 · 2024-10-04T03:17:42Z

Describe the feature request

Support to running onnx models in webgpu using deno

Describe scenario use case

Actually not working because this error:
error: Uncaught (in promise) Error: no available backend found. ERR: [webgpu] backend not found.

Debug
console.log(await navigator.gpu.requestAdapter())

GPUAdapter {
  features: GPUSupportedFeatures [
    "depth-clip-control",
    "timestamp-query",
    "indirect-first-instance",
    "shader-f16",
    "depth32float-stencil8",
    "texture-compression-bc",
    "rg11b10ufloat-renderable",
    "bgra8unorm-storage",
    "float32-filterable",
    "texture-format-16-bit-norm",
    "texture-adapter-specific-format-features",
    "pipeline-statistics-query",
    "timestamp-query-inside-passes",
    "mappable-primary-buffers",
    "texture-binding-array",
    "buffer-binding-array",
    "storage-resource-binding-array",
    "sampled-texture-and-storage-buffer-array-non-uniform-indexing",
    "uniform-buffer-and-storage-texture-array-non-uniform-indexing",
    "partially-bound-binding-array",
    "multi-draw-indirect",
    "multi-draw-indirect-count",
    "push-constants",
    "address-mode-clamp-to-zero",
    "address-mode-clamp-to-border",
    "polygon-mode-line",
    "polygon-mode-point",
    "conservative-rasterization",
    "vertex-writable-storage",
    "clear-texture",
    "spirv-shader-passthrough",
    "multiview",
    "shader-f64",
    "shader-i16",
    "shader-primitive-index",
    "shader-unused-vertex-output"
  ],
  limits: GPUSupportedLimits {
    maxTextureDimension1D: 16384,
    maxTextureDimension2D: 16384,
    maxTextureDimension3D: 2048,
    maxTextureArrayLayers: 2048,
    maxBindGroups: 8,
    maxBindingsPerBindGroup: 1000,
    maxBufferSize: 2147483647,
    maxDynamicUniformBuffersPerPipelineLayout: 16,
    maxDynamicStorageBuffersPerPipelineLayout: 8,
    maxSampledTexturesPerShaderStage: 8388606,
    maxSamplersPerShaderStage: 8388606,
    maxStorageBuffersPerShaderStage: 8388606,
    maxStorageTexturesPerShaderStage: 8388606,
    maxUniformBuffersPerShaderStage: 8388606,
    maxUniformBufferBindingSize: 2147483648,
    maxStorageBufferBindingSize: 2147483648,
    minUniformBufferOffsetAlignment: 32,
    minStorageBufferOffsetAlignment: 32,
    maxVertexBuffers: 16,
    maxVertexAttributes: 32,
    maxVertexBufferArrayStride: 2048,
    maxInterStageShaderComponents: 128,
    maxColorAttachments: 8,
    maxColorAttachmentBytesPerSample: 32,
    maxComputeWorkgroupStorageSize: 65536,
    maxComputeInvocationsPerWorkgroup: 1024,
    maxComputeWorkgroupSizeX: 1024,
    maxComputeWorkgroupSizeY: 1024,
    maxComputeWorkgroupSizeZ: 1024,
    maxComputeWorkgroupsPerDimension: 65535
  },
  info: GPUAdapterInfo {
    vendor: "4098",
    architecture: "",
    device: "29695",
    description: "AMD Radeon RX 6600 (RADV NAVI23)"
  },
  isFallbackAdapter: false
}

The text was updated successfully, but these errors were encountered:

fs-eire · 2024-10-07T18:42:44Z

Thank you for reporting this issue. since navigator.gpu is supported in deno, it should work. could you share me reproduce steps (or a repo) so that I can debug it?

jlucaso1 · 2024-10-07T19:16:23Z

@fs-eire Try the following:

import { InferenceSession } from "npm:onnxruntime-web";

await InferenceSession.create(
  "https://huggingface.co/briaai/RMBG-1.4/resolve/main/onnx/model_quantized.onnx",
  {
    executionProviders: ["webgpu"],
  }
);

I'm using deno 2.0.0-rc.10, but deno stable version get the same error.

fs-eire · 2024-10-07T19:53:00Z

I am not sure about this “npm:onnxruntime-web” syntax. In a nodejs based web app project I used “onnxruntime-web/webgpu” (“webgpu” is the export name that defined in package.json). If I just import “onnxruntime-web” it doesn’t load webgpu support.

jlucaso1 · 2024-10-07T20:36:51Z

@fs-eire I understand.
I got a different error when i use import { InferenceSession } from "npm:onnxruntime-web/webgpu"; on stable versions of Deno:

failed to asynchronously prepare wasm: both async and sync fetching of the wasm failed
Aborted(both async and sync fetching of the wasm failed)
error: Uncaught (in promise) Error: no available backend found. ERR: [webgpu] RuntimeError: Aborted(both async and sync fetching of the wasm failed). Build with -sASSERTIONS for more info.

The release candidate version of Deno throw the same error:

error: Uncaught (in promise) Error: no available backend found. ERR: [webgpu] backend not found.
    at resolveBackendAndExecutionProviders (file:///run/media/jlucaso/secondary/projects/webgpu/node_modules/.deno/[email protected]/node_modules/onnxruntime-common/dist/esm/backend-impl.js:120:15)
    at async Function.create (file:///run/media/jlucaso/secondary/projects/webgpu/node_modules/.deno/[email protected]/node_modules/onnxruntime-common/dist/esm/inference-session-impl.js:180:52)

fs-eire · 2024-10-07T22:09:21Z

It seems that Emscripten does not have well support for Deno. The generated javascript glue file didn't work well in Deno:

The feature detection may be working incorrectly.

var ENVIRONMENT_IS_WEB = typeof window == "object";
var ENVIRONMENT_IS_WORKER = typeof importScripts == "function";
var ENVIRONMENT_IS_NODE = typeof process == "object" && typeof process.versions == "object" && typeof process.versions.node == "string";
var ENVIRONMENT_IS_SHELL = !ENVIRONMENT_IS_WEB && !ENVIRONMENT_IS_NODE && !ENVIRONMENT_IS_WORKER;

There is no special treatment for Deno so further operation may be unexpected.

Loading from file URL is explicitly disallowed:

  if (isFileURI(wasmBinaryFile)) {
    err(`warning: Loading from a file URI (${wasmBinaryFile}) is not supported in most browsers. See https://emscripten.org/docs/getting_started/FAQ.html#how-do-i-run-a-local-webserver-for-testing-why-does-my-program-stall-in-downloading-or-preparing`);
  }

(after I fixed the above 2 locally) For some reason, the wasm factory Promise does not return.
I have no idea how to fix this one. I think there is not much that I can do before Emscripten officially support Deno.

fs-eire · 2024-10-07T22:28:16Z

Update: OK I hacked the Emscripten glue JS and finally get into Deno's webgpu. And now I am blocked with a bug in Deno's webgpu implementation:

adapter.limits.maxBufferSize's value is 18446744073709551615n. This is not expected (adapter.limits.maxBufferSize should be number but not BigInt) and will cause an error later:

Cannot convert a BigInt value to a number

link to the issue: denoland/deno#22029

fs-eire · 2024-10-08T00:17:52Z

I don't know why maxBufferSize on my devbox is bigint. will later change to a different machine and try.

fs-eire · 2024-10-08T05:11:04Z

changed another PC and the value of maxBufferSize is still 18446744073709551615n.

mattvr · 2024-10-11T00:04:44Z

According to the WebGPU spec, maxBufferSize is an unsigned long long, so I believe returning a BigInt is correct.

jlucaso1 · 2024-10-11T02:21:22Z

In google chrome adapter.limits.maxBufferSize returns a number:

adapter.limits.maxBufferSize
2147483648
typeof adapter.limits.maxBufferSize
'number'

Chrome: 129.0.6668.100
OS: Linux

fs-eire · 2024-10-11T04:26:59Z

I am not sure about the details but at least the implementation of Deno WebGPU is incorrect.

It maxBufferSize is considered to be number, then adapter.limits.maxBufferSize being bigint is incorrect.

It maxBufferSize is considered to be bigint, then using it in adapter.requestDevice( { requiredLimits : { maxBufferSize: adapter.limits.maxBufferSize }}) should not run into error Cannot convert a BigInt value to a number.

jlucaso1 · 2024-10-12T00:21:54Z

@fs-eire what deno version you tested early?
I'm using deno 2.0.0 stable and getting a normal number in adapter.limits.maxBufferSize

const adapter = await navigator.gpu.requestAdapter();

console.log(adapter?.limits.maxBufferSize); // 2147483647

const device = await adapter?.requestDevice( { requiredLimits : { maxBufferSize: adapter.limits.maxBufferSize! }})

console.log(device) // GPUDevice with no error

fs-eire · 2024-10-12T07:00:09Z

I tried both 1.46.3 and 2.0.0

deno repl --unstable-webgpu

Deno 1.46.3
exit using ctrl+d, ctrl+c, or close()
> const adapter = await navigator.gpu.requestAdapter();
undefined
> console.log(adapter?.limits.maxBufferSize);
18446744073709551615n
undefined

deno repl --unstable-webgpu

Deno 2.0.0
exit using ctrl+d, ctrl+c, or close()
> const adapter = await navigator.gpu.requestAdapter();
undefined
> console.log(adapter?.limits.maxBufferSize);
18446744073709551615n
undefined
>

fs-eire · 2024-11-27T11:11:34Z

The blocking issue denoland/deno#22029 is fixed in upstream. I will check if it works in latest version of deno.

fs-eire · 2024-11-27T12:48:41Z

seems the fix is not yet in latest (canary) version.

jlucaso1 · 2024-11-28T00:13:22Z

@fs-eire i think the fix is in the latest canary now

ry · 2024-12-06T03:27:29Z

Fix has been released in Deno 2.1.2

jlucaso1 · 2024-12-06T11:04:42Z

@fs-eire @ry
Updating to Deno to 2.1.3 i'm getting the same error:

import { InferenceSession } from "npm:onnxruntime-web";

await InferenceSession.create(
  "https://huggingface.co/briaai/RMBG-1.4/resolve/main/onnx/model_quantized.onnx",
  {
    executionProviders: ["webgpu"],
  }
);

deno run --unstable-webgpu main.ts

error: Uncaught (in promise) Error: no available backend found. ERR: [webgpu] backend not found.
    at resolveBackendAndExecutionProviders (file:///run/media/jlucaso/secondary/projects/webgpu/node_modules/.deno/[email protected]/node_modules/onnxruntime-common/dist/esm/backend-impl.js:120:15)
    at async Function.create (file:///run/media/jlucaso/secondary/projects/webgpu/node_modules/.deno/[email protected]/node_modules/onnxruntime-common/dist/esm/inference-session-impl.js:180:52)
    at async file:///run/media/jlucaso/secondary/projects/webgpu/main.ts:3:1

jlucaso1 added the feature request request for unsupported feature or enhancement label Oct 4, 2024

github-actions bot added ep:WebGPU ort-web webgpu provider platform:web issues related to ONNX Runtime web; typically submitted using template labels Oct 4, 2024

fs-eire mentioned this issue Oct 8, 2024

[Feature Request] #22317

Closed

av mentioned this issue Oct 24, 2024

[webgpu] backend not found when use onnxruntime-web denoland/deno#25397

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature request] Deno webgpu support #22309

[Feature request] Deno webgpu support #22309

jlucaso1 commented Oct 4, 2024

fs-eire commented Oct 7, 2024

jlucaso1 commented Oct 7, 2024

fs-eire commented Oct 7, 2024 •

edited

Loading

jlucaso1 commented Oct 7, 2024

fs-eire commented Oct 7, 2024

fs-eire commented Oct 7, 2024 •

edited

Loading

fs-eire commented Oct 8, 2024

fs-eire commented Oct 8, 2024 •

edited

Loading

mattvr commented Oct 11, 2024

jlucaso1 commented Oct 11, 2024

fs-eire commented Oct 11, 2024

jlucaso1 commented Oct 12, 2024 •

edited

Loading

fs-eire commented Oct 12, 2024 •

edited

Loading

fs-eire commented Nov 27, 2024

fs-eire commented Nov 27, 2024

jlucaso1 commented Nov 28, 2024

ry commented Dec 6, 2024

jlucaso1 commented Dec 6, 2024

[Feature request] Deno webgpu support #22309

[Feature request] Deno webgpu support #22309

Comments

jlucaso1 commented Oct 4, 2024

Describe the feature request

Describe scenario use case

fs-eire commented Oct 7, 2024

jlucaso1 commented Oct 7, 2024

fs-eire commented Oct 7, 2024 • edited Loading

jlucaso1 commented Oct 7, 2024

fs-eire commented Oct 7, 2024

fs-eire commented Oct 7, 2024 • edited Loading

fs-eire commented Oct 8, 2024

fs-eire commented Oct 8, 2024 • edited Loading

mattvr commented Oct 11, 2024

jlucaso1 commented Oct 11, 2024

fs-eire commented Oct 11, 2024

jlucaso1 commented Oct 12, 2024 • edited Loading

fs-eire commented Oct 12, 2024 • edited Loading

fs-eire commented Nov 27, 2024

fs-eire commented Nov 27, 2024

jlucaso1 commented Nov 28, 2024

ry commented Dec 6, 2024

jlucaso1 commented Dec 6, 2024

fs-eire commented Oct 7, 2024 •

edited

Loading

fs-eire commented Oct 7, 2024 •

edited

Loading

fs-eire commented Oct 8, 2024 •

edited

Loading

jlucaso1 commented Oct 12, 2024 •

edited

Loading

fs-eire commented Oct 12, 2024 •

edited

Loading