From 791621b0225c0c6fafb49004809ca25da54ddbad Mon Sep 17 00:00:00 2001 From: aciddelgado Date: Sun, 5 Nov 2023 07:57:49 -0800 Subject: [PATCH] docs --- docs/ContribOperators.md | 16 ++++++---------- docs/OperatorKernels.md | 2 +- 2 files changed, 7 insertions(+), 11 deletions(-) diff --git a/docs/ContribOperators.md b/docs/ContribOperators.md index 646465ef8b56f..13129c0a67a82 100644 --- a/docs/ContribOperators.md +++ b/docs/ContribOperators.md @@ -2393,19 +2393,15 @@ This version of the operator has been available since version 1 of the 'com.micr #### Attributes
-
is_past_bsnh : int
-
Whether past kv uses BSNH, otherwise BNSH. Default value is 1 (BSNH).
kv_num_heads : int (required)
Number of attention heads for k and v
num_heads : int (required)
Number of attention heads for q
scale : float
Custom scale will be used if specified. Default value is 1/sqrt(head_size)
-
unidirectional : int
-
Whether every token can only attend to previous tokens. Default value is 1.
-#### Inputs (3 - 6) +#### Inputs
query : T
@@ -2418,8 +2414,8 @@ This version of the operator has been available since version 1 of the 'com.micr
past state key with support for format BSNH or BNSH. When past_key uses same tensor as present_key(k-v cache), it is of length max_sequence_length... otherwise of length past_sequence_length.
past_value (optional) : T
past state value with support for format BSNH or BNSH. When past_value uses same tensor as present_value(k-v cache), it is of length max_sequence_length... otherwise of length past_sequence_length.
-
past_sequence_length (optional) : M
-
When buffered past_key and past_value is used (present_key uses same tensor as past_key), requiredto specify past_sequence_length (could be 0). Otherwise, past_sequence_length inferred from past_key.
+
attention_mask : M
+
2d Tensor of shape (batch_size, past_sequence_length + sequence_length). Must be a right padding mask.
#### Outputs @@ -2438,8 +2434,8 @@ This version of the operator has been available since version 1 of the 'com.micr
T : tensor(float16)
Constrain input and output to float tensors.
-
M : tensor(int32), tensor(int64)
-
Constrain past sequence length to int tensor.
+
M : tensor(int64)
+
Constrain mask to int tensor.
@@ -4950,7 +4946,7 @@ This version of the operator has been available since version 1 of the 'com.micr ### **com.microsoft.RotaryEmbedding** - RotaryEmbedding is the implementation of rotary positional embeddings (RoPE). The positions are represented as rotation matrices + RotaryEmbedding is the implementation of rotary positional embeddings (RoPE). The positions are represented as rotation matrices that are multiplied to query and key before the inner product of query and key is taken. #### Version diff --git a/docs/OperatorKernels.md b/docs/OperatorKernels.md index bbfc33a915fae..b3e0dbe5bd98a 100644 --- a/docs/OperatorKernels.md +++ b/docs/OperatorKernels.md @@ -835,7 +835,7 @@ Do not modify directly.* |GreedySearch|*in* input_ids:**I**
*in* max_length:**I**
*in* min_length:**I**
*in* repetition_penalty:**T**
*in* vocab_mask:**I**
*in* prefix_vocab_mask:**I**
*in* attention_mask:**I**
*out* sequences:**I**|1+|**T** = tensor(float), tensor(float16)| |GridSample|*in* X:**T1**
*in* Grid:**T1**
*out* Y:**T2**|1+|**T1** = tensor(float)
**T2** = tensor(float)| |GroupNorm|*in* X:**T**
*in* gamma:**M**
*in* beta:**M**
*out* Y:**T**|1+|**T** = tensor(float), tensor(float16)| -|GroupQueryAttention|*in* query:**T**
*in* key:**T**
*in* value:**T**
*in* past_key:**T**
*in* past_value:**T**
*in* past_sequence_length:**M**
*out* output:**T**
*out* present_key:**T**
*out* present_value:**T**|1+|**M** = tensor(int32), tensor(int64)
**T** = tensor(float16)| +|GroupQueryAttention|*in* query:**T**
*in* key:**T**
*in* value:**T**
*in* past_key:**T**
*in* past_value:**T**
*in* attention_mask:**M**
*out* output:**T**
*out* present_key:**T**
*out* present_value:**T**|1+|**M** = tensor(int64)
**T** = tensor(float16)| |Inverse|*in* X:**T**
*out* Y:**T**|1+|**T** = tensor(double), tensor(float), tensor(float16)| |Irfft|*in* X:**T**
*out* Y:**T**|1+|**T** = tensor(double), tensor(float), tensor(float16)| |LongformerAttention|*in* input:**T**
*in* weight:**T**
*in* bias:**T**
*in* mask:**T**
*in* global_weight:**T**
*in* global_bias:**T**
*in* global:**G**
*out* output:**T**|1+|**T** = tensor(float), tensor(float16)|