[Web] InferenceSession.create returns number as exception with model generated by torch.multinomial

[cmdli]

Describe the issue

When trying to load a model that uses torch.multinomial, the Web runtime returns an error that is just a number. I would expect either this to be supported or for the runtime to return a more helpful error.

Console output:

Uncaught (in promise) 8457208

To reproduce

Not all this code may be necessary, but this is a dummy case I created that reproduces the issue:

Pytorch code:

import torch.nn as nn
import torch

class TestModule(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, input):
        return torch.multinomial(input, num_samples=1)
    
model = TestModule()
dummy_input = torch.zeros((16,), dtype=torch.long)
export_options = torch.onnx.ExportOptions(dynamic_shapes=True)
onnx_model = torch.onnx.dynamo_export(model, dummy_input, export_options=export_options)
onnx_model.save("test.onnx")

JS code:

import * as ort from "onnxruntime-web";

const session = await ort.InferenceSession.create("test.onnx", {
    executionProviders: ["wasm"],
});

Urgency

This isn't urgent as I suspect that there might be reasons to not support torch.multinomial, but it did eat up a few hours of my time so I would argue it should be fixed eventually for the next person.

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.1

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

[baijumeswani]

@fs-eire @satyajandhyala could you please help with this?

[fs-eire]

"onnxruntime-web" has removed the exception handling in Release build to reduce binary size. This is why the Uncaught (in promise) <a-number> error will show.

Usually this is because the model or the model input is invalid. To validate this, you can simply change the JS code to use onnxruntime-node instead of onnxruntime-web, and replace the "wasm" backend to "cpu":

import * as ort from "onnxruntime-node";

const session = await ort.InferenceSession.create("test.onnx", {
    executionProviders: ["cpu"],
});

Run this code in Node.js and then you get the following error:

Error: Load model from test.onnx failed:Node (aten_multinomial_0) Op (aten_multinomial) [ShapeInferenceError] (op_type:Log, node name: n5): input typestr: T, has unsupported type: tensor(int64)
    at new OnnxruntimeSessionHandler (D:\pg\2024-03-18\node_modules\onnxruntime-node\dist\backend.js:24:92)
    at D:\pg\2024-03-18\node_modules\onnxruntime-node\dist\backend.js:64:29
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)

It looks like a shape inference error is occurred.

[cmdli]

That makes sense to me. Would it be possible to include just a short error message to clue the user that exception handling requires either NodeJS or (presumably) the Debug build? I was assuming that I was doing something wrong with the model, but because there weren't any clues in the error message it took me awhile to narrow it down to just this one Pytorch function.

[justinchuby]

side: The aten_multinomial function in PyTorch exporter doesn't handle long inputs for now. Only floating point values are tested.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.