com.microsoft.MultiHeadAttention is unsupported

[music-dino]

ROCm#3425

[marko-fabo-htec]

The computation details of the Multi-Head Attention can be found in this paper:
https://arxiv.org/abs/1706.03762

[marko-fabo-htec]

An example about how to implement the behavior of the MultiHeadAttention operator:
microsoft/onnxruntime#19924

Useful articles about Transformers and Attention:
https://towardsdatascience.com/transformers-explained-visually-part-1-overview-of-functionality-95a6dd460452

[marko-fabo-htec]

Input description: https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/contrib_ops/cpu/bert/multihead_attention_helper.h#L267

[marko-fabo-htec]

PR: ROCm#3650

[marko-fabo-htec]

benchmark results:

               |     REF     |     CPU     |     GPU     |
mha            | 0.539452 ms | 0.841959 ms | 0.270837 ms |
mha_cross      | 0.440712 ms | 0.753624 ms | 0.285106 ms |
mha_kv_packed  | 1.121980 ms | 1.454690 ms | 0.268043 ms |
mha_qkv_packed | 1.375370 ms | 1.740890 ms | 0.264313 ms |