Have a question about model

[haeju-poza]

Hi, at your custom/layers.py, there are EncoderLayer and DecoderLayer.
you actually use EncoderLayer, which is solely self-attention.

in DecoderLayer, the forward function receives encode_out, which means DecoderLayer follows the vanilla transformer structure, I think.

In the Music Transformer paper, I remember that this uses only self-masked attention at decoder part.

Did you try using the vanilla transformer structure with relative attention for implementation?
How was the result compared to using only self-masked attention, if you tried?