Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RUNTIME_EXCEPTION occurs when running a per-channel quantized model in QDQ format #17718

Closed
feihongfiona opened this issue Sep 27, 2023 · 1 comment
Labels
quantization issues related to quantization

Comments

@feihongfiona
Copy link

Describe the issue

I have a per-channel quantized model in QDQ format. Running the model gets the following terminal interrupt :

onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running DequantizeLinear node. Name:'W_DequantizeLinear/duplicated' Status Message: /private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_a92wmqsrpo/croot/onnxruntime_1689679134480/work/onnxruntime/core/providers/cpu/quantization/quantize_linear.cc:36 void onnxruntime::PrepareForQDQ(const onnxruntime::TensorShape &, const onnxruntime::Tensor &, const onnxruntime::Tensor *, int64_t, int64_t &, int64_t &, int64_t &) scale.Shape().NumDimensions() == 1 && scale.Shape()[0] == broadcast_dim was false. scale must be 1D tensor with size 525

I trace the source code bellow:

template <typename T>
class DequantizeLinear final : public OpKernel {
 public:
  DequantizeLinear(const OpKernelInfo& info) : OpKernel(info) {
    if (!info.GetAttr<int64_t>("axis", &axis_).IsOK()) {
      axis_ = 1;   
    }
  }

  Status Compute(OpKernelContext* context) const override;

 private:
  int64_t axis_;
};

I think the axis_ should default to 0

To reproduce

Conv_layer_quant_U8U8_perch_sym01_QDQ.onnx.zip

Urgency

No response

Platform

Mac

OS Version

13.5

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.15.1

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

@github-actions github-actions bot added the quantization issues related to quantization label Sep 27, 2023
@wschin
Copy link
Contributor

wschin commented Oct 2, 2023

Default is 1 per ONNX standard, so onnxruntime shouldn't change its behavior. If you think this behavior is wrong, please file an issue to ONNX. If your model is generated by a quantization tool, please file a bug to that tool since it generates an invalid model. Or, as a workaround, you can edit your model file and fix that attribute.

@wschin wschin closed this as completed Oct 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
quantization issues related to quantization
Projects
None yet
Development

No branches or pull requests

2 participants