additional gains from QDQ

[mengniwang95]

Hi,

I am confused about what additional gains we can get from QDQ format compared with quantization with QLinearOps, can you share me some ideas? For example, if QDQ format is more general, which cases confirm it?

Thanks

[yufenglee]

Comparing with QLinearOps, QDQ format is much more flexible and helps the ONNX quantization ecosystem. Here are some benefits examples:

• different EPs/accelerators supports different set of quantized operators. As long as they support QuantizeLinear and DeQuantizeLinear, they can run the model successfully.
• ONNX exporter (like pytorch to onnx) and converter (like tf2onnx) can convert/export quantized models from original framework to quantized ONNX model with QDQ format.

[BIT-jwq]

Hello Yufeng,

@yufenglee I am still curious to learn more about this topic.

1. Is the advantage of this format solely related to supporting different EPs? Are there any other benefits associated with this format?

2. Where can the theory paper or official demonstrations be related to the QDQ format?

I would appreciate your comments on this matter.

In fact, I am planning to conduct comparative experiments to validate the advantages of the QDQ format.

Could you please provide some advice and share your comments on this?

[Rickustc]

Hello Yufeng,
@yufenglee I am working in generate Qoperator format,but torch can only export QDQ format,is there any solution for QDQ to Qoperator ?
thanks

[JiliangNi]

Hello Yufeng, @yufenglee I am working in generate Qoperator format,but torch can only export QDQ format,is there any solution for QDQ to Qoperator ? thanks

@Rickustc Did you find any solution for QDQ to Qoperator? I met the similar problem and I found another quant tool which generates Qoperator model.

[UsingtcNower]

Hello Yufeng, @yufenglee I am working in generate Qoperator format,but torch can only export QDQ format,is there any solution for QDQ to Qoperator ? thanks

@Rickustc Did you find any solution for QDQ to Qoperator? I met the similar problem and I found another quant tool which generates Qoperator model.

I have the same question. @Rickustc @JiliangNi Did you find the solution?

[yufenglee]

Please refer to #21137 for the answer.