Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

turn on neural_speed by default #19627

Merged
merged 7 commits into from
Mar 20, 2024
Merged

turn on neural_speed by default #19627

merged 7 commits into from
Mar 20, 2024

Conversation

yufenglee
Copy link
Member

Description

the crash caused by the neural_speed turns out to be a very corn case. Turn it on by default.

Motivation and Context

yihonglyu
yihonglyu previously approved these changes Feb 23, 2024
@snnn
Copy link
Member

snnn commented Feb 23, 2024

In my opinion the problem still should be fixed, especially we see the fix is there but for what reason we refuse doing so? Windows and Office are deployed to billions of devices. So definitely some people will hit the issue. Even if they only represent a small percentage of the total population, we will still see a huge number of crash reports.
Besides, xbyak is also a new dependency which may also have compatibility issues. This is the first time we introduce a JIT engine in the core framework and the code will be deployed billions of devices. Should we be more careful on that? Anyone peeked into xbyak's code? xbyak does a lot of non-trivial things. I would think this is the biggest change we ever had in ONNX Runtime in the past 6 years.

@yufenglee
Copy link
Member Author

In my opinion the problem still should be fixed, especially we see the fix is there but for what reason we refuse doing so? Windows and Office are deployed to billions of devices. So definitely some people will hit the issue. Even if they only represent a small percentage of the total population, we will still see a huge number of crash reports. Besides, xbyak is also a new dependency which may also have compatibility issues. This is the first time we introduce a JIT engine in the core framework and the code will be deployed billions of devices. Should we be more careful on that? Anyone peeked into xbyak's code? xbyak does a lot of non-trivial things. I would think this is the biggest change we ever had in ONNX Runtime in the past 6 years.

the fix is in progress. At the same time, we don't want user to experience very slow experience for the corner case.

@yufenglee yufenglee force-pushed the yufeng/turnon_neural_speed branch from 04d63bf to 0f5cf75 Compare March 15, 2024 00:55
@yufenglee yufenglee requested a review from a team as a code owner March 15, 2024 00:55
@yufenglee yufenglee force-pushed the yufeng/turnon_neural_speed branch from 0f5cf75 to f12a66d Compare March 15, 2024 03:45
@yufenglee yufenglee merged commit 15219e2 into main Mar 20, 2024
95 checks passed
@yufenglee yufenglee deleted the yufeng/turnon_neural_speed branch March 20, 2024 19:49
TedThemistokleous pushed a commit to TedThemistokleous/onnxruntime that referenced this pull request May 7, 2024
### Description
<!-- Describe your changes. -->
the crash caused by the neural_speed turns out to be a very corn case.
Turn it on by default.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants