Skip to content

Commit

Permalink
update comment
Browse files Browse the repository at this point in the history
  • Loading branch information
PeixuanZuo committed Feb 1, 2024
1 parent f793e4f commit f1f00a0
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ def _make_causal_mask(
"""
Make causal mask used for Attention with attribute unidirectional == 1.
The mask is a upper triangular matrix with shape [sequence_length, total_sequence_length].
Putting a 1 indicates that the token at this position should be masked.
For Example:
sequence_length = 5, total_sequence_length = 5,
mask: [[0. 1. 1. 1. 1.]
Expand Down

0 comments on commit f1f00a0

Please sign in to comment.