[Performance] Perf decrease

[P0man]

Describe the issue

I run cpu model (can't provide it) with different versions of onnxruntime (builded from source). v1.11.1, v1.14.0, v1.17.0.
And timings of inference are

version
mean min max q95

1.11.1
328970 303519 367536 348662

1.14.0
351867 338643 401872 382355

1.17.0
348458 328082 380977 376152

Inference c++ code is the same.
So I don't really get whats the problem with performance decreasing with new versions of onnxruntime

To reproduce

Sorry cannot share model

Urgency

No response

Platform

Linux

OS Version

rh8.8

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.11.0; 1.14.0; 1.17.0

ONNX Runtime API

C++

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Model File

No response

Is this a quantized model?

No

[tianleiwu]

Try profiling: https://onnxruntime.ai/docs/performance/tune-performance/profiling-tools.html.

Report back if you have some findings (like some operators or part takes longer in 1.11 vs 1.17).

[skottmckay]

Also not clear how exactly you're measuring performance and whether external things are being included in the timing.

• run the model once as a warmup.
  • this will initialize a lot of infrastructure so the following calls to run are faster
  • measure the calls after this
• measure only the call to run
  • inputs should not be allocated/populated within the measurement
  • no print statements or anything else should be in the measurement
• make sure you're using a high resolution timer
  • what is best depends on the platform/performance measurement code

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.