-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problems about groups #19
Comments
The 'groups' are not like the groups for group convolutions. In this case, each of the multi-attention heads takes in the full input dimension (not 1/n_groups of it). Computationally, this is done as one convolution and then reshaped to separate the heads. |
However, it seems like not to separate |
Seems like |
According to the listed code lines, it seems that self.groups has no effect on the result. I beleive no matter how to set self.groups, the code can only realize the case "group=1". |
Should not the key_conv, query_conv and value_conv be defined with group parameters? I am not clear what group parameters do in the following lines.
Stand-Alone-Self-Attention/attention.py
Lines 43 to 50 in a983f0f
The text was updated successfully, but these errors were encountered: