Skip to content

Commit

Permalink
refine code
Browse files Browse the repository at this point in the history
  • Loading branch information
inkcherry committed Aug 23, 2024
1 parent 3ebb373 commit fc0ec67
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions deepspeed/sequence/layer.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ def uneven_heads_all2all(input, scatter_idx, gather_idx, group):
if scatter_idx < 2:
# The compatibility handling of 4D and 3D tensors, standardizing to 3D.
input = input.view(input.shape[0], input.shape[1], -1)

seq_len, batch_size, h=input.shape

input = input.reshape(seq_len*h, batch_size)
Expand Down

0 comments on commit fc0ec67

Please sign in to comment.