[Graph optimization] INVALID_GRAPH error returned by InferenceSession with offline optimization

[jenchaine]

Describe the issue

We tried to use offline optimization to reduce startup time as explained here:
https://onnxruntime.ai/docs/performance/model-optimizations/graph-optimizations.html

But the initialization of the optimized model fails with INVALID_GRAPH error.
check_model returns the same error.
However, online optimization works fine.

To reproduce

1. Serialize the optimized model

options = onnxruntime.SessionOptions()
options.optimized_model_filepath = "optimized.onnx"
options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_ENABLE_ALL

sess = onnxruntime.InferenceSession("model.onnx", providers=["CPUExecutionProvider"], sess_options=options)

2. Load the optimized model without optimization

options = onnxruntime.SessionOptions()
options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
sess = onnxruntime.InferenceSession("optimized.onnx", providers=["CPUExecutionProvider"], sess_options=options)

The following error will be returned:

10 : INVALID_GRAPH : Load model from optimized.onnx failed:This is an invalid model. In Node, (\"\", If, \"\", -1) : (\"if_node\": tensor(bool),) -> (\"if_out\",) , Error Graph must be in single static assignment (SSA) form, however '_inlfunc_SequenceMap_token_0_SequenceMap_out_sequence_0_seqempty' has been used as output names multiple times."

Urgency

We plan to deploy the model in Q2 2025.
With online graph optimization, it takes about 5 seconds to initialize the session, which is quite slow if we need to create the session frequently.

Platform

Windows

OS Version

10

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.2

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[tianleiwu]

@jenchaine, Please share the onnx model for reproducing the issue.

[jenchaine]

Hello @tianleiwu,
Sorry I cannot share the model because of confidential issues.
I will avoid using the SequenceMap (see issue #23024) so normally I will no longer have this issue.
I have also noticed some strange bahavior: the InferenceSession initialization is faster with optimization on ... 5 seconds (on) v.s. 7 seconds (off).