How to calculate the relative positional embeddings from a row offset and column offset?

[songkq]

Thanks for sharing the great idea.
When I read the paper, I have some issues about how to compute the relative positional embeddings, i.e. r_(a-i, b-j) , from a row offset a-i and a column offset b-j.
Is there any explicit formula for the calculating process?
Looking forward to your reply.

[leaderj1001]

Thanks for your comments !
The paper is implemented referring to the following paper.

Attention Augmented Convolutional Networks Link

Thank you.

[Jimmy880]

@leaderj1001
Hello, I wonder whether you implement the relative position embedding in the selfattention.
I notice that you just exploit two random tensors to represent position in h and w direction while the paper said "The row and column offsets are associated with an embedding r_{a−i} and r_{b−j}".
I am confused about whether the so called "embedding" should be implemented as the nn.Embedding operation in pytorch.

[siyuan2018]

@leaderj1001
Hello, I wonder whether you implement the relative position embedding in the selfattention.
I notice that you just exploit two random tensors to represent position in h and w direction while the paper said "The row and column offsets are associated with an embedding r_{a−i} and r_{b−j}".
I am confused about whether the so called "embedding" should be implemented as the nn.Embedding operation in pytorch.

Hi, I am having the same question here. Did you figure out how to compute the relative position embedding？Thanks