Inference fails with libc++abi terminating error

[pjjajal]

Describe the issue

Inference with an ONNX model results in libc++abi terminating message and my shell terminating the process. Screenshot:

When attempting this on a different machine we get the following error:

OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
fish: Job 1, 'python test.py' terminated by signal SIGABRT (Abort)

And upon setting the environment variable we get the following error:

libc++abi: terminating
fish: Job 1, 'python test.py' terminated by signal SIGABRT (Abort)

The ONNX model used was originally a TensorFlow model that was converted to ONNX using the tensorflow-onnx model converter (tf2onnx). The original model (TensorFlow) was created using the nnsmith model generator. The TensorFlow model was converted as follows python -m tf2onnx.convert --saved-model <path to tfnet> --output ./model.onnx --opset 16

To reproduce

Install Dependencies:
pip install onnxruntime
pip install torch

Steps:

1. Download the ONNX model.
2. Download the inputs and the output names (download to the same directory as the onnx model).
3. Run test.py from the same directory as the files.

Test file:

import onnxruntime as ort
import numpy as np
import torch

ort_inputs = torch.load("./ort_inputs.pt")
output_names = torch.load("./output_names.pt")
print(ort.get_available_providers())
sess = ort.InferenceSession("./model.onnx")
ort_outputs = sess.run(output_names=output_names, input_feed=ort_inputs)

Urgency

No response

Platform

MacOS

OS Version

13.5.2 (22G91)

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.15.1

ONNX Runtime API

Python

Architecture

ARM64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[pjjajal]

I made a mistake in the original issue. The platform is macOS.

[snnn]

The error comes from libiomp5 but we don't use openmp at all.

[pjjajal]

The error comes from libiomp5 but we don't use openmp at all.

The error message referring to openmp comes from a co-worker who attempted to reproduce it. When I run the same script I get the following:

[snnn]

Could you please build ONNX Runtime from source in debug mode and give me a stacktrace ? Here is the instructions: https://onnxruntime.ai/docs/build/inferencing.html

After that, you just install the newly built package and run

lldb -- python test.py

It will give us everything.

[pjjajal]

I built the latest ONNX Runtime from source and attempted this again. I am not getting the error on the latest build, but it still occurs with v1.15.1.

I can attempt to build v1.15.1 from source and get the stack trace for that. I was having a bit of trouble with that build.

[snnn]

We just published 1.16.0. Can you try that one instead?

[yuslepukhin]

We have not shipped with openmp for some time.

[pjjajal]

The model works fine with 1.16.0. I am getting no errors.

What originally caused the issue in 1.15.1?

[snnn]

Sorry I don't know. Without a stack trace I cannot tell what the issue is. And even if we know, we cannot go back to patch it. We only provide bug fixes to the latest release.