[Node.js] Support for loading models with external data in onnxruntime-node

[rotemdan]

Describe the issue

When I try to load a model with external data using onnxruntime-node, I get:

Exception during initialization: C:\a\_work\1\s\onnxruntime\core\opt
imizer\initializer.cc:43 onnxruntime::Initializer::Initializer [ONNXRuntimeError] : 1 : FAIL : t
ensorprotoutils.cc:792 onnxruntime::utils::GetExtDataFromTensorProto External initializer: posit
ional_embedding offset: 0 size to read: 7680000 given file_length: 2293760 are out of bounds or
can not be read in full.

External files are located at the same directory as the .onnx file. I also tried positioning the Node.js script file at the same directory as the model (current working directory), and still getting the same error. The model loads fine in Python.

I found existing issues about support in onnxruntime-web, like #13006, #17151 and the recent pull request #18535 that attempts to address it.

However, I found no mention on support for large models in onnxruntime-node. Does it already exist? I haven't managed to get it to work, and there is no hint on the Node.js API or its source code that external files are loaded automatically?

Since onnxruntime-node is based on C (NAPI) binding, I believe it's not bound by the limitations of WASM? So, if it's not currently supported, could it be added?

To reproduce

import Onnx from 'onnxruntime-node'

await Onnx.InferenceSession.create('some-large-model.onnx')

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.16.3

Execution Provider

Other / Unknown

[akj10]

Hello, I am also hitting this issue with a model size > 2GB, when I try to run it in C++ environment.
The model is SDXL-Turbo: https://huggingface.co/stabilityai/sdxl-turbo/tree/main/unet
Were you able to find any workaround for this?