QNN EP: Context cache only works with single partition

[john-dance]

Describe the issue

The context cache

qnnOptions["qnn_context_cache_enable"] = "1";
qnnOptions["qnn_context_cache_path"] = contextCache;

only works if the graph can be executed in a single partition.

onnxruntime + QNN Execution Provider + libQnnHtp.so backend.

If more than one partition is needed, onnxruntime throws an exception in qnn_execution_provider. This restriction does not exist in other environments such as TfLite + QNN delegate.

  if (num_of_partitions > 1) {
    ORT_ENFORCE(!context_cache_enabled_, "Only support single partition for context cache feature.");
  }

To reproduce

Run the model attached to #18353 with the following options:

qnnOptions["backend_path"] = "<path>/libQnnHtp.so";
qnnOptions["profiling_level"] = "basic";
qnnOptions["htp_performance_mode"] = "burst";
qnnOptions["qnn_context_cache_enable"] = "1";
qnnOptions["qnn_context_cache_path"] = "<valid cache path>";
onnxSessionOptions.AppendExecutionProvider("QNN", qnnOptions);

Urgency

No response

Platform

Android

OS Version

34

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.16

ONNX Runtime API

C++

Architecture

ARM64

Execution Provider

Other / Unknown

Execution Provider Library Version

QNN: 2.16

[jywu-msft]

yes, this is a current limitation that we will need to address.

[HectorSVC]

Here's the PR:
#18865

[HectorSVC]

Please give it a try with latest main branch.

[john-dance]

Will do. Thank you.

[john-dance]

@HectorSVC - The attached model now runs in a single partition. I'm getting a model that requires multiple partitions before actually closing this issue. Thank you for your work on this.

[HectorSVC]

confirmed it worked