[QNN] Initialization failure in particular use case.

[JohnnyWang19921007]

Describe the issue

When we try to run a loop to initializing, model inferring and finalizing on QNN 2.25.0 engine (with onnxruntime 1.18.1) every 3~5 sec, and there is an error coming out in initial stage.

To reproduce

[Test case 1]
QNN : 2.25
Onnxruntime: 1.18.1

[Test case 2]
QNN : 2.25
Onnxruntime: 1.19.0

• Reproduce steps with test app:

referred to onnxruntime-inference-examples/c_cxx/QNN_EP/mobilenetv2_classification at main · microsoft/onnxruntime-inference-examples · GitHub
We only change to do “run_ort_qnn_ep” for 10000 times to simulate particular use case.

• Test case 1

It will come out an error as below in a few seconds:

[onnxruntime] [] sequential_executor.cc:516 onnxruntime::ExecuteKernel: Non-zero status code returned while running QNN_13825321319279089785_0 node. Name:'QNNExecutionProvider_QNN_13825321319279089785_0_0' Status Message: QNN graph execute error. Error code: 1100
run failed : Non-zero status code returned while running QNN_13825321319279089785_0 node. Name:'QNNExecutionProvider_QNN_13825321319279089785_0_0' Status Message: QNN graph execute error. Error code: 1100

or

[onnxruntime] [] sequential_executor.cc:514 onnxruntime::ExecuteKernel: Non-zero status code returned while running QNN_8567928183500563082_0 node. Name:'QNNExecutionProvider_QNN_8567928183500563082_0_0' Status Message: QNN graph execute error. Error code: 6001
run failed : Non-zero status code returned while running QNN_8567928183500563082_0 node. Name:'QNNExecutionProvider_QNN_8567928183500563082_0_0' Status Message: QNN graph execute error. Error code: 6001

• Test case 2

We have passed the loop test.

Just want to confirm that this issue has been fixed in onnxruntime v1.19.0

Urgency

It's a production issue for our customer, will pending the release plan and impact shipping date.

Platform

Windows

OS Version

Windows 11

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.0 and 1.18.1

ONNX Runtime API

C++

Architecture

ARM64

Execution Provider

Other / Unknown

Execution Provider Library Version

QNN 2.25.0 HTP backend

[jywu-msft]

just wanted to confirm that you are bringing your own version of QNN.
ORT.QNN 1.18.1 (nuget and pypi) packages shipped with QNN 2.22.10
were these tests working on a previous version of ORT.QNN? if so, which version(s) of ORT and QNN?

[JohnnyWang19921007]

Hi @jywu-msft

We have changed the way we finalize the session from session->release() to session.reset() and this issue cannot be reproduced anymore.