Skip to content

Commit

Permalink
[GPU] fix sdpa invalid args: Do not use indirect sdpa when it has no …
Browse files Browse the repository at this point in the history
…itermediate buffer
  • Loading branch information
ahnyoung-paul committed Nov 29, 2024
1 parent 7b3fd79 commit 74f06ff
Showing 1 changed file with 4 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,10 @@ struct scaled_dot_product_attention_impl : multi_stage_primitive<scaled_dot_prod
if (!instance.has_indirect_inputs())
return false;

// if instance has no intermediate buffer, uses only default_sdpa
if (instance.get_intermediates_memories().empty())
return false;

const auto& params = *instance.get_impl_params();
const auto indirect_axis = desc->indirect_axis;
if (params.input_layouts[get_beam_table_id(desc)].get_partial_shape()[indirect_axis].get_length() == 1)
Expand Down

0 comments on commit 74f06ff

Please sign in to comment.