Extend Attention Bias Broadcast Support

[tianleiwu]

Description

Previously, MultiHeadAttention supports relative position bias of shape [1, N, S, T] or [B, N, S, T], and DecoderMaskedMultiHeadAttention supports [1, N, S, T]. This will extend the support to allow [1, N, S, T], [B, N, S, T], [B, 1, S, T] and [1, 1, S, T] for CUDA and CPU EPs.

• Rename the input of "relative position bias" to "attention bias" because it can also be used for other types of bias, like ALiBi (Attention with Linear Biases) or attention mask.
• Update unfused kernel to support broadcasting 2nd dimension of attention bias.
• Update efficient attention to support broadcasting 2nd dimension of attention bias.
• Update operators (MultiHeadAttention, DecoderMaskedMultiHeadAttention, Attention, PackedAttention, PackedMultiHeadAttention) to support broadcast attention bias on CUDA and CPU EPs.
• Update ROCm, DML and WebGPU naming to be consistent. (Note that those EPs do not support broadcasting attention_bias for now).
• Add attention bias tests for MultiHeadAttention.
• Update operator documents
• Update benchmark script

Other changes:

• Fix some checks in multihead-attention.ts
• Add helper functions to dump tensors given dimensions.

Motivation and Context