From 8ea3e68192967314f80ef4e7abd21a59b7cfd722 Mon Sep 17 00:00:00 2001 From: Jeff Bloomfield Date: Thu, 4 Jan 2024 10:10:46 -0800 Subject: [PATCH] Update ContribOperators.md --- docs/ContribOperators.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/ContribOperators.md b/docs/ContribOperators.md index 131db5d8d9b37..38fceef67de25 100644 --- a/docs/ContribOperators.md +++ b/docs/ContribOperators.md @@ -155,6 +155,8 @@ This version of the operator has been available since version 1 of the 'com.micr
Corresponding past and present are same tensor, its size is (2, batch_size, num_heads, max_sequence_length, head_size)
qkv_hidden_sizes : list of ints
Hidden dimension of Q, K, V: hidden_size, hidden_size and v_hidden_size
+
rotary_embedding_dim : int
+
Dimension of rotary embedding. Limited to 32, 64 or 128. Default value is head_size
scale : float
Custom scale will be used if specified. Default value is 1/sqrt(head_size)
unidirectional : int