[LayerNormalization Operator] Why LayerNormalization do not support weight and bias auto broadcast.

[triple-Mu]

Describe the issue

Input is [1, 16, 256, 256]
I want to do layernorm in axes 1 and the weight and bias shape is [1, 16, 1, 1].
It will broadcast to [1, 16, 256, 256].
But the error is:

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running LayerNormalization node. Name:'/LayerNormalization' Status Message: Size of X.shape()[axis:] == 1048576. Size of scale and bias (if provided) must match this. Got scale size of 16 and bias size of 16

So what can i do?

To reproduce

Urgency

No response

Platform

Linux

OS Version

ubuntu 20.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.16.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

cpu

Model File

No response

Is this a quantized model?

No

[tianleiwu]

It's valid case according to onnx spec This operator supports unidirectional broadcasting (tensors Scale and B should be unidirectional broadcastable to tensor X).

ORT does not support it yet since the original implementation for ms domain (before the operator added to onnx domain) does not support broadcasting.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[github-actions]

This issue has been automatically closed due to inactivity. Please reactivate if further support is needed.