virtual void toC::Slice::resolve(): Assertion `false' failed.

[edesaras]

Hi, I am trying to export BERT to C file but, unfortunately, i get the error in the title:

I am using Colab and i install onnx2c with the following snippet

import os
!apt install libprotobuf-dev protobuf-compiler
!git clone https://github.com/kraiskil/onnx2c.git
os.chdir("onnx2c")
!git submodule update --init
os.mkdir("build")
os.chdir("build")
!cmake -DCMAKE_BUILD_TYPE=DEBUG ..
!make onnx2c

Download the model weights from Huggingface
!wget https://huggingface.co/google-bert/bert-base-uncased/resolve/main/model.onnx

Convert to C
!./onnx2c model.onnx > bert-base-uncased.c

Last few lines of the ONNX2C LOG:

2024-04-08 14-48-25.218 [Trace] (tryResolveNode)   Making sure node attributes are in the graph
2024-04-08 14-48-25.218 [Trace] (addTensor) Updating existing tensor: bert.embeddings.position_ids
2024-04-08 14-48-25.218 [Trace] (addTensor)   was:   bert.embeddings.position_ids:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 512 }  buffer 0x591eca27b190
2024-04-08 14-48-25.218 [Trace] (addTensor)   new:   bert.embeddings.position_ids:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 512 }  buffer 0x591eca27b190
2024-04-08 14-48-25.218 [Trace] (addTensor)   now:   bert.embeddings.position_ids:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 512 }  buffer 0x591eca27b190
2024-04-08 14-48-25.219 [Trace] (addTensor) Updating existing tensor: /bert/embeddings/Constant_1_output_0
2024-04-08 14-48-25.219 [Trace] (addTensor)   was:   /bert/embeddings/Constant_1_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca305660
2024-04-08 14-48-25.219 [Trace] (addTensor)   new:   /bert/embeddings/Constant_1_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca305660
2024-04-08 14-48-25.219 [Trace] (addTensor)   now:   /bert/embeddings/Constant_1_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca305660
2024-04-08 14-48-25.219 [Trace] (addTensor) Updating existing tensor: /bert/embeddings/Unsqueeze_output_0
2024-04-08 14-48-25.219 [Trace] (addTensor)   was:   /bert/embeddings/Unsqueeze_output_0:  gen 1  init 0  IO 0  const 0  recurs 0  dims { 1 1 }  buffer 0
2024-04-08 14-48-25.219 [Trace] (addTensor)   new:   /bert/embeddings/Unsqueeze_output_0:  gen 1  init 0  IO 0  const 0  recurs 0  dims { 1 1 }  buffer 0
2024-04-08 14-48-25.219 [Trace] (addTensor)   now:   /bert/embeddings/Unsqueeze_output_0:  gen 1  init 0  IO 0  const 0  recurs 0  dims { 1 1 }  buffer 0
2024-04-08 14-48-25.219 [Trace] (addTensor) Updating existing tensor: /bert/embeddings/Constant_2_output_0
2024-04-08 14-48-25.219 [Trace] (addTensor)   was:   /bert/embeddings/Constant_2_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca305d70
2024-04-08 14-48-25.219 [Trace] (addTensor)   new:   /bert/embeddings/Constant_2_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca305d70
2024-04-08 14-48-25.219 [Trace] (addTensor)   now:   /bert/embeddings/Constant_2_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca305d70
2024-04-08 14-48-25.219 [Trace] (addTensor) Updating existing tensor: /bert/embeddings/Constant_3_output_0
2024-04-08 14-48-25.219 [Trace] (addTensor)   was:   /bert/embeddings/Constant_3_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca3060a0
2024-04-08 14-48-25.219 [Trace] (addTensor)   new:   /bert/embeddings/Constant_3_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca3060a0
2024-04-08 14-48-25.322 [Trace] (addTensor)   now:   /bert/embeddings/Constant_3_output_0:  gen 1  init 1  IO 0  const 1  recurs 0  dims { 1 }  buffer 0x591eca3060a0
2024-04-08 14-48-25.322 [Trace] (tryResolveNode)    (end of attribute-input-vectors)
2024-04-08 14-48-25.322 [Trace] (tryResolveNode) Resolving node
2024-04-08 14-48-25.322 [Fatal] (resolve) Non-const inputs to Slice not handled
onnx2c: /content/onnx2c/src/nodes/slice.h:63: virtual void toC::Slice::resolve(): Assertion `false' failed.
/bin/bash: line 1:  2357 Aborted                 (core dumped) ./onnx2c model.onnx > bert-base-uncased.c

[kraiskil]

Hi,

The error in the title is just a convenience when building onnx2c for debugging (easier to stop with the debugger at the problem spot), and the default logging level is a bit verbose in the debug build... The real issue is on the previous line: [Fatal] (resolve) Non-const inputs to Slice not handled.

As a general rule, onnx2c doesn't handle inference time defined tensor dimensions. This is because since the size of the tensor is not known at compile time, allocating the memory would have to be at run time, which would make the generated code less suitable for microcontrollers.

Bert seems to be a language model - run time definitions of tensor sizes is probably pretty unavoidable when doing natural language procesing. I'm afraid there is nothing much to do here - onnx2c is probably the wrong tool for this.

[john-ye1]

Excuse me, so this mean that the onnx2c tool cannot support operators related to tensor dimension changing well? I am using an onnx model with two convolutional layers, but I encountered this error, and it also generated a error with the concat dimension mismatch. Thank you.

[john-ye1]

so can you recommend me some other tools that can achieve this function？Thank you very much.

[kraiskil]

What is not implemented is having a tensor size depend on the values in the input data. If this is unavoidable in your application/NN design, then I guess you would need a bigger target system that could run e.g. onnx-runtime.

I am using an onnx model with two convolutional layers, but I encountered this error, and it also generated a error with the concat dimension mismatch.

There is something strange here that I don't understand.
Two convolutional layers alone should not be able to produce dynamically sized tensors. Also, onnx2c exits on the first error, so it should not be possible to see both this error and a error with concat.

For the Slice node, there is not much to do.
So please have closer look at the error you are seeing. If you are not using dynamically sized tensors, or the problem is not with the Slice node, please open a new issue to figure it out.

[john-ye1]

sorry,I haven't describe this question clearly.I encounter the error with slice when I use the onnx2c transforming the onnx of yolov8 to C,but when I transformer the two convolutional layers to C,I encounter the error with the concat. The model of the Conv is same as the examples of mnist basically, but the only difference is that the convolution is quantized convolution .I have uploaded the onnx architecture of the model .Thank you

[kraiskil]

Ok, clearly this problem is not related from the 'Slice' node :) And I don't see any reason for dynamic sized tensors here.

Please open a new issue with either the .onnx file or the trace output (onnx2c -l4) not to derail this issue.