Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix AMD pipeline test failures #19250

Merged
merged 5 commits into from
Jan 24, 2024
Merged

Fix AMD pipeline test failures #19250

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3096988
Update multihead_attention.cu
wangyems Jan 23, 2024
19ffe27
Update multihead_attention.h
wangyems Jan 23, 2024
67be406
Update multihead_attention.cu
wangyems Jan 23, 2024
63efcf4
Update multihead_attention.cu
wangyems Jan 23, 2024
a8e0dce
Update multihead_attention.cu
wangyems Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions onnxruntime/contrib_ops/rocm/bert/multihead_attention.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@
scale_ = info.GetAttrOrDefault<float>("scale", 0.0f);

past_present_share_buffer_ = info.GetAttrOrDefault<int64_t>("past_present_share_buffer", 0LL) != 0LL;
is_unidirectional_ = info.GetAttrOrDefault<int64_t>("unidirectional", 0) == 1;

using HipT = typename ToHipType<T>::MappedType;
using AttentionTunableOp = GemmSoftmaxGemmPermuteTunableOp<HipT>;
Expand Down Expand Up @@ -121,8 +122,8 @@
query, key, value, bias,
key_padding_mask, relative_position_bias,
past_key, past_value, past_seq_len,
&attn,
num_heads_, mask_filter_value_, scale_,
&attn, num_heads_,

Check warning on line 125 in onnxruntime/contrib_ops/rocm/bert/multihead_attention.cu

View workflow job for this annotation

GitHub Actions / cpplint

[cpplint] onnxruntime/contrib_ops/rocm/bert/multihead_attention.cu#L125 

 Line ends in whitespace.  Consider deleting these extra spaces.  [whitespace/end_of_line] [4]
Raw output 
onnxruntime/contrib_ops/rocm/bert/multihead_attention.cu:125:  Line ends in whitespace.  Consider deleting these extra spaces.  [whitespace/end_of_line] [4]
mask_filter_value_, scale_, false, /*is_unidirectional_*/

Check warning on line 126 in onnxruntime/contrib_ops/rocm/bert/multihead_attention.cu

View workflow job for this annotation

GitHub Actions / cpplint

[cpplint] onnxruntime/contrib_ops/rocm/bert/multihead_attention.cu#L126 

 Line ends in whitespace.  Consider deleting these extra spaces.  [whitespace/end_of_line] [4]
Raw output 
onnxruntime/contrib_ops/rocm/bert/multihead_attention.cu:126:  Line ends in whitespace.  Consider deleting these extra spaces.  [whitespace/end_of_line] [4]
past_present_share_buffer_, false, device_prop.maxThreadsPerBlock));

if (attn_type_ == kDecoderMaskedMultiHeadAttention && attn.sequence_length != 1) {
Expand Down
1 change: 1 addition & 0 deletions onnxruntime/contrib_ops/rocm/bert/multihead_attention.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class MultiHeadAttention final : public RocmKernel {
float mask_filter_value_;
float scale_;
bool past_present_share_buffer_{false};
bool is_unidirectional_{false};

// type-erased GemmSoftmaxGemmPermuteTunableOp<HipT>, the reason for this is:
// 1. We don't want to include the cuh file where GemmSoftmaxGemmPermuteTunableOp<HipT> is defined.
Expand Down
Loading