You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Scaling values (eg. alpha and beta) and float constants (eg. epsilon) are used by our kernels and passed from the user through the Device Op API. The following is a list of how those values are used in C.K
alpha and beta are used in Reduction and Softmax
epsilon is used in BatchNorm and LayerNorm
averageFactor is used in BatchNorm
And currently, different component might use different methods to define the types of those values at the Device Op API layer. A unified specification to define these values is necessary to make our Device Op API stable and brings less potential issues.
The following is my suggested method to specify the types of scaling values and float constants:
At each Device Op API, the arguments of scaling and float constant values are declared to be double
Inside the Device Op codes, the scaling and float constant values are converted to some AccDataType and saved as the AccDataType variables
At each kernel interface, the arguments of scaling and float constant values are declared to be some AccDataType
The text was updated successfully, but these errors were encountered:
Scaling values (eg.
alpha
andbeta
) and float constants (eg.epsilon
) are used by our kernels and passed from the user through the Device Op API. The following is a list of how those values are used in C.KAnd currently, different component might use different methods to define the types of those values at the Device Op API layer. A unified specification to define these values is necessary to make our Device Op API stable and brings less potential issues.
The following is my suggested method to specify the types of scaling values and float constants:
double
AccDataType
and saved as theAccDataType
variablesAccDataType
The text was updated successfully, but these errors were encountered: