[Wait for #2794, #2795][Graph] add inplace setting through layer property #2796
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The method of setting the inplace-type has been redefined. The reason why inplace processing becomes complicated is that since a multi-out layer shares output variables, so it needs to be considered whether or not inplace can be performed. To simplify the problem, the layers that can perform inplace even after the multi-out layer are only no-operation layers(no-op layers). These no-op layers include identity, reshape, and flatten layers. For other layers, even if they support inplace, they cannot perform inplace when there is a multi-out layer in front of them. Note that because no-op layers connected with multi-out layer share memory with the multi-out layer, so they have the same properties as the multi-out layer. This is expressed as RESTRICTING in our script. Based on these definitions, I've redesigned the method of setting inplace type. 1. By default, initialize the inplace type for each layer. If supportInPlace is true, it will be initialized as NON_RESTRICTING; otherwise, it will be initialized as NONE. 2. However, not all layers are initialized like this. For multi-out layers or no-op layers, if supportInPlace is true, they will be initialized as RESTRICTING types(However, the no-op layer will be changed to a non-restricting type if that is not connected with the multi-out layer). 3. After initialization, confirm the input connections from the network_graph.cpp to determine the final inplace type. It's clearer to see the source code for this part. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
- The variables with the same meaning are written as "in_place" or "is_inplace" by the script, so I unified it to use the term "is_inplace". - Some layer's finalize function includes code that determines whether or not to support in-place depending on the tensor type. However, this code does not work. The reason it seems like this code is working is because there is a similar purpose of code at the top of the `canExecuteInPlace` function within the `network_graph.cpp` and that code works. It is meaningless to determine whether or not to support in-place within the `finalize` function because the `canExecuteInPlace` function, which decides how InPlace will behave, is called before the `finalize` function. The canExecuteInPlace function is called during `compile` while the finalize function is called during `initialize` after `compile`. Therefore, setting supportInplace inside the finalize function does not work. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
baek2sm
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
gichan-jang,
anyj0527,
lhs8928,
songgot,
jihochu,
DonghakPark,
SeoHyungjun,
skykongkong8,
djeong20 and
EunjuYang
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2796. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
- now you can set in-place flag through layer property of tensor operation layers. - rename "initializeInPlaceType" func to "initializeInPlace". now "is_inplace" property is set in that function, too. - in some layers, support_backwarding flag may be changed by the in-place setting. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
baek2sm
changed the title
skykongkong8
approved these changes
Nov 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
operation layers.
"is_inplace" property is set in that function, too.
in-place setting.
Self evaluation:
Signed-off-by: Seungbaek Hong [email protected]