HuggingFace failure #7877

sroussey · 2023-12-28T02:08:48Z

What version of Bun is running?

1.0.20

What platform is your computer?

Darwin 23.2.0 arm64 arm

What steps can reproduce the bug?

import { pipeline } from "@xenova/transformers";

const generateEmbedding = await pipeline(
  "feature-extraction",
  "Supabase/gte-small"
);

console.log(generateEmbedding);

What is the expected behavior?

Something not an error...

What do you see instead?

TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)
      at new Worker (node:worker_threads:124:27)
      at yc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7890)
      at Cc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7945)
      at lt (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:5687)
      at Et (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:9720)
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])
❯ bun -v
1.0.20
❯ uname -mprs
Darwin 23.2.0 arm64 arm

Additional information

No response

The text was updated successfully, but these errors were encountered:

sroussey · 2023-12-28T02:16:56Z

Slightly more detail

22 |         __classPrivateFieldSet(this, _OnnxruntimeSessionHandler_inferenceSession, new binding_1.binding.InferenceSession(), "f");
23 |         if (typeof pathOrBuffer === 'string') {
24 |             __classPrivateFieldGet(this, _OnnxruntimeSessionHandler_inferenceSession, "f").loadModel(pathOrBuffer, options);
25 |         }
26 |         else {
27 |             __classPrivateFieldGet(this, _OnnxruntimeSessionHandler_inferenceSession, "f").loadModel(pathOrBuffer.buffer, pathOrBuffer.byteOffset, pathOrBuffer.byteLength, options);
                 ^
error: Error
      at new OnnxruntimeSessionHandler (/Users/steve/Code/elmers/node_modules/onnxruntime-node/dist/backend.js:27:13)
      at /Users/steve/Code/elmers/node_modules/onnxruntime-node/dist/backend.js:64:29
      at processTicksAndRejections (:61:77)

Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. 
✖ DATA [FAILED: BuildMessage: ModuleNotFound resolving "undefined" (entry point)]
1 | /*!
2 | * ONNX Runtime Web v1.14.0
3 | * Copyright (c) Microsoft Corporation. All rights reserved.
4 | * Licensed under the MIT License.
5 | */


[[[[[ lots of minified code]]]]]


TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)
      at new Worker (node:worker_threads:124:27)
      at yc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7890)
      at Cc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7945)
      at lt (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:5687)
      at Et (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:9720)
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])

sroussey · 2023-12-28T17:39:39Z

When debugging, before the line is question is even run, I have a breakpoint and use the debugger to look at things.

I type _OnnxruntimeSessionHandler_inferenceSession into the debug console and get this:

I click on the weakmap to open:

I then click on OnnxruntimeSessionHandler:InferenceSession and get this:

Just looking at the properties segfaults the process (and causes the WebSocket closed message).

sroussey · 2023-12-28T17:57:51Z

Digging around this seems to be the culprit, but it makes no sense to me:

1 | export default "native";

Exception has occurred.
TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)
    at Worker (native)
    at new Worker (node:worker_threads:124:14)
    at yc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:252:32)
    at Cc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:254:45)
    at lt (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:196:24)
    at Et (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:332:147)
    at <?>.wasm-function[10917] (native)
    at <?>.wasm-function[1580] (native)
    at <?>.wasm-function[2786] (native)
    at <?>.wasm-function[5903] (native)
    ^
error: Session is not initialized.

sroussey · 2023-12-29T04:46:14Z

I went to the onnx which is where the errors are happening. I found a simple example that uses node:

https://github.com/microsoft/onnxruntime-inference-examples/blob/main/js/api-usage_inference-session/inference-session-create.js

It works fine.

However...

When I change it to require onnxruntime-web instead of onnxruntime-web bun gives me almost the same error!

Weird.

I get this error:

failed to create inference session: TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)

Run the same thing in node and I get:

failed to create inference session: TypeError [ERR_WORKER_PATH]: The worker script or module filename must be an absolute path or a relative path starting with './' or '../'. Received "blob:nodedata:be2e77f3-2beb-4779-845a-4ff54d158079"

That is obviously a different error. And it has to deal with Workers.

sroussey · 2023-12-29T15:51:28Z

There is an issue for the above:

microsoft/onnxruntime#18933

sroussey · 2023-12-29T16:19:24Z

Note here: "It's a known issue that onnxruntime-web multi-thread not loading in node.js."

microsoft/onnxruntime#17274 (comment)

"@xenova/transformers" is surely using the web version and not the node version and thus the cause of this issue.

[Edit: this is false, it determines which to use and correctly does so.]

sroussey · 2023-12-29T18:46:04Z

OK, it was dealing with workers and wasm as a backup if the native code failed. ONNX on node with WASM can only use one thread, so I have a one line fix for "@xenova/transformers". With that fix, it appears to work just fine but it is much slower than it should be.

So the bug is a failure in the native code that works fine with node but fails with bun.

Things run slower as it runs the native code first, loads the model and bails, then loads the wasm, loads the model, and succeeds. On a browser it can use multiple threads. So the browser version is faster than bun and the node version is faster than bun unless bun can run the native module (assuming using my fix).

sroussey · 2023-12-29T22:37:53Z

I went through building onnx runtime and the node build both in debug and found everything to work and not error.

Long story short, onnx has already fixed the issue but "xenova/transformers" is using an old version.

sroussey · 2023-12-29T22:48:47Z

Main fix: huggingface/transformers.js#488

Nnadivictory25 · 2024-01-12T21:47:39Z

heyy man @sroussey what worked for you ? I am currently getting this error in bun trying to run embeddings , please I really need help , this has me stuck for a few days now

sroussey · 2024-01-12T21:49:48Z

Use my version temporarily

@sroussey/transformers

MariuzM · 2024-02-02T17:41:19Z

@sroussey thank you i had same issue huggingface/transformers.js#558 your version fixed

sroussey · 2024-02-02T19:35:42Z

It's just temporary until fixed upstream.

sroussey · 2024-02-02T19:35:59Z

But glad you have it working!

TimPietrusky · 2024-03-12T12:04:15Z

As mentioned in huggingface/transformers.js#489 it should be solved in the v3 branch of transformers.js!

EdamAme-x · 2024-08-12T10:25:01Z

hmm...
same problem occurred for now

sroussey added the bug Something isn't working label Dec 28, 2023

Electroid added the needs investigate Needs to be investigated to find the root cause label Dec 28, 2023

sroussey mentioned this issue Dec 29, 2023

Update onnxruntime to recent version (works with bun) huggingface/transformers.js#489

Closed

jimlloyd mentioned this issue Feb 4, 2024

Unknown model class "nomic_bert", attempting to construct from base class. huggingface/transformers.js#562

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HuggingFace failure #7877

HuggingFace failure #7877

sroussey commented Dec 28, 2023 •

edited

Loading

sroussey commented Dec 28, 2023 •

edited

Loading

sroussey commented Dec 28, 2023

sroussey commented Dec 28, 2023 •

edited

Loading

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023 •

edited

Loading

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023

Nnadivictory25 commented Jan 12, 2024

sroussey commented Jan 12, 2024

MariuzM commented Feb 2, 2024

sroussey commented Feb 2, 2024

sroussey commented Feb 2, 2024

TimPietrusky commented Mar 12, 2024

EdamAme-x commented Aug 12, 2024

HuggingFace failure #7877

HuggingFace failure #7877

Comments

sroussey commented Dec 28, 2023 • edited Loading

What version of Bun is running?

What platform is your computer?

What steps can reproduce the bug?

What is the expected behavior?

What do you see instead?

Additional information

sroussey commented Dec 28, 2023 • edited Loading

sroussey commented Dec 28, 2023

sroussey commented Dec 28, 2023 • edited Loading

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023 • edited Loading

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023

sroussey commented Dec 29, 2023

Nnadivictory25 commented Jan 12, 2024

sroussey commented Jan 12, 2024

MariuzM commented Feb 2, 2024

sroussey commented Feb 2, 2024

sroussey commented Feb 2, 2024

TimPietrusky commented Mar 12, 2024

EdamAme-x commented Aug 12, 2024

sroussey commented Dec 28, 2023 •

edited

Loading

sroussey commented Dec 28, 2023 •

edited

Loading

sroussey commented Dec 28, 2023 •

edited

Loading

sroussey commented Dec 29, 2023 •

edited

Loading