fix: inplace operation and avoid for loop #72
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Nice to meet you. Thanks for the great work and sharing.
I have been using your SCINet and have found a few bugs and inefficient implementations.
I have been using it with local fixes, but I thought I would contribute to the community. If you would like to make use of it, please feel free to do so.
There are two areas that need to be fixed.
The first is to fix the use of in-place operation in the Tensor calculation.
Pytorch does not allow in-place operations such as +=, but they were being used in several places.
This operation is only used when the model has certain arguments and may have been missed during testing.
The second is the implementation of SCINet_Tree.zip_up_the_pants.
This implementation uses a for loop for the Tensor operation, which does not take advantage of the GPU's computational efficiency.
Therefore, I modified the implementation to use only matrix operations without changing the calculation result.
In my experiments on my local GPU, this modification resulted in a speedup of about 30~40 times.