Check padding density by input of embedding module #19821
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
guyang3532
force-pushed
the
yangu/check_padding_input
branch
from
fded099 to
372ab79
Compare
pengwa
added
the
training
guyang3532
force-pushed
the
yangu/check_padding_input
branch
2 times, most recently
from
6568886 to
3d8614a
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Fixed
Show fixed
Hide fixed
guyang3532
force-pushed
the
yangu/check_padding_input
branch
2 times, most recently
from
f3eb16d to
6027f03
Compare
pengwa
reviewed
Mar 12, 2024
orttraining/orttraining/core/optimizer/compute_optimizer/padding_elimination.cc
Outdated
Show resolved
Hide resolved
orttraining/orttraining/core/optimizer/compute_optimizer/padding_elimination.cc
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_training_manager.py
Outdated
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
guyang3532
force-pushed
the
yangu/check_padding_input
branch
from
6027f03 to
280f127
Compare
guyang3532
force-pushed
the
yangu/check_padding_input
branch
from
280f127 to
0beaed7
Compare
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Fixed
Show fixed
Hide fixed
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Fixed
Show fixed
Hide fixed
pengwa
reviewed
Apr 10, 2024
orttraining/orttraining/core/optimizer/compute_optimizer/padding_elimination.cc
Outdated
Show resolved
Hide resolved
pengwa
reviewed
Apr 10, 2024
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Show resolved
Hide resolved
pengwa
reviewed
Apr 10, 2024
orttraining/orttraining/python/training/ortmodule/_graph_execution_manager.py
Outdated
Show resolved
Hide resolved
pengwa
reviewed
Apr 10, 2024
pengwa
reviewed
Apr 10, 2024
orttraining/orttraining/core/optimizer/compute_optimizer/padding_elimination.cc
Outdated
Show resolved
Hide resolved
pengwa
previously approved these changes
Apr 10, 2024
pengwa
approved these changes
Apr 10, 2024
TedThemistokleous
pushed a commit
to TedThemistokleous/onnxruntime
that referenced
this pull request
May 7, 2024
### Description
The PaddingElimination optimization is enabled when the density of
embedding padding less than 90%. We need to check the density of the
embedding padding to decide whether enable the optimization.
Before this pr, we just check the inputs of graph and correlate one with
the embedding node by iterate graph from the embedding node back to one
graph input.
This is hard to be general because there may be complicated pattern
between graph input and embedding node.
This pr check padding density by the direct input of embedding module
rather than the input of graph at the first graph execution when
exporting onnx graph.
And if the density < 90%, insert a flag PythonOp after the embedding
node as:
```
Embedding
|
PythonOp (func_name:_FlagPaddingElimination) (insert if density < 90%)
|
Following graph
```
When the PaddingElimination is invoked, it check if there is the flag
PythonOp(func_name:_FlagPaddingElimination) after the Embedding node and
if it is, remove it and do the padding elimination optimization.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The PaddingElimination optimization is enabled when the density of embedding padding less than 90%. We need to check the density of the embedding padding to decide whether enable the optimization.
Before this pr, we just check the inputs of graph and correlate one with the embedding node by iterate graph from the embedding node back to one graph input.
This is hard to be general because there may be complicated pattern between graph input and embedding node.
This pr check padding density by the direct input of embedding module rather than the input of graph at the first graph execution when exporting onnx graph.
And if the density < 90%, insert a flag PythonOp after the embedding node as:
When the PaddingElimination is invoked, it check if there is the flag PythonOp(func_name:_FlagPaddingElimination) after the Embedding node and if it is, remove it and do the padding elimination optimization.