fix shape inference bug

[inisis]

Description

for nodes like add, their input should be merged dynamically

Motivation and Context

when doing shape inference, for nodes like Add, currently when doing _onnx_infer_single_node, their inputs are generated from last node's output, but they should be merged.

[inisis]

@microsoft-github-policy-service agree
@microsoft-github-policy-service agree company="tsingmicro"

[inisis]

@tianleiwu Can you please review it

[tianleiwu]

/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline

[tianleiwu]

/azp run Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Android CI Pipeline

[tianleiwu]

/azp run iOS CI Pipeline,ONNX Runtime React Native CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 10 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 10 pipeline(s).

[tianleiwu]

Please add some unit tests in https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/python/onnxruntime_test_python_symbolic_shape_infer.py

[inisis]

Please add some unit tests in https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/python/onnxruntime_test_python_symbolic_shape_infer.py

yes, there is a failed case, where can I upload the onnx model, it's about 400MB, and it's a commonly known model

[tianleiwu]

Please add some unit tests in https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/python/onnxruntime_test_python_symbolic_shape_infer.py

yes, there is a failed case, where can I upload the onnx model, it's about 400MB, and it's a commonly known model

We usually add some tiny model in testing. Like a model with only one MatMul node that could trigger the code path.

400MB is too large. You can create an issue and add a reproduce script and link to the model in that issue.

[inisis]

Please add some unit tests in https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/python/onnxruntime_test_python_symbolic_shape_infer.py

yes, there is a failed case, where can I upload the onnx model, it's about 400MB, and it's a commonly known model

We usually add some tiny model in testing. Like a model with only one MatMul node that could trigger the code path.

400MB is too large. You can create an issue and add a reproduce script and link to the model in that issue.

#19870 Hi, issue is raised

[tianleiwu]

#19870 Hi, issue is raised

See my comments in the issue. I can run shape inference with --auto_merge flag successfully without this change. Could you double check that?

[inisis]

#19870 Hi, issue is raised

See my comments in the issue. I can run shape inference with --auto_merge flag successfully without this change. Could you double check that?

Hi, the uploaded model is incorrect, could you please redownload it, I have checked it myself.

[inisis]

@tianleiwu Can you please review it.

[tianleiwu]

/azp run Windows GPU CI Pipeline,ONNX Runtime Web CI Pipeline,Linux GPU CI Pipeline,orttraining-amd-gpu-ci-pipeline,Android CI Pipeline,iOS CI Pipeline,ONNX Runtime React Native CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 7 pipeline(s).

[inisis]

Hi @tianleiwu How can I fix the failing cases, the error is not clear to me.

[tianleiwu]

Let me re-run the failed pipeline

[inisis]

@tianleiwu Hi, can this be merged, I have many more commits to push.

[inisis]

@tianleiwu since shape inference is super important when doing model optimization, can I keep this code in my repo, I would prefer maintain my own version of onnxruntime shape inference, and I will commit my code gradually, there are lots of bugs.

[tianleiwu]

There is one required pipeline "Windows GPU CI Pipeline" failed so this cannot merge. Let me try re-run last time. If failed again, please try rebase to latest main.

[inisis]

There is one required pipeline "Windows GPU CI Pipeline" failed so this cannot merge. Let me try re-run last time. If failed again, please try rebase to latest main.

it falied again, and it seems to be like a problem with cmake

[tianleiwu]

/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux GPU CI Pipeline,Linux OpenVINO CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-ortmodule-distributed,Big Models

[azure-pipelines]

Azure Pipelines successfully started running 10 pipeline(s).

[inisis]

@tianleiwu the failed case show that Failed to download Python from the Github Actions python registry (https://github.com/actions/python-versions), maybe the network problem.

[tianleiwu]

/azp run Windows x64 QNN CI Pipeline,Windows GPU CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline

[tianleiwu]

/azp run onnxruntime-binary-size-checks-ci-pipeline,Android CI Pipeline,iOS CI Pipeline,ONNX Runtime React Native CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 10 pipeline(s).