Static quantization crashes with "TypeError: iteration over a 0-d array"

[onnxruntime-user]

Describe the issue

During static quantization with percentile calibration, the following crash happens with onnxruntime-gpu 1.17.0:

Collecting tensor data and making histogram ...
Traceback (most recent call last):
File ".\new_onnx_quantize_static.py", line 21, in
oq.quantize_static(preprocessed_path, quantized_path, Mock(),
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\quantize.py", line 496, in quantize_static
calibrator.collect_data(calibration_data_reader)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 546, in collect_data
self.collector.collect(clean_merged_dict)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 724, in collect
return self.collect_absolute_value(name_to_arr)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 739, in collect_absolute_value
dtypes = set(a.dtype for a in arr)
TypeError: iteration over a 0-d array

Older onnxruntime versions did not have this issue.

To reproduce

Model is: minimal_model.zip. Code to reproduce:

import onnxruntime.quantization as oq
import numpy as np


class Mock:
    def __init__(self):
        self.i = 0

    def get_next(self):
        if self.i > 10:
            return None
        self.i += 1
        return {"input": np.random.randint(0, 255, size=(1, 3, 32, 32), dtype=np.uint8)}

if __name__ == "__main__":
    onnx_path = "minimal_model.onnx"
    preprocessed_path = onnx_path[:-5] + "_preprocessed.onnx"
    quantized_path = onnx_path[:-5] + "_quantized.onnx"
    oq.quant_pre_process(onnx_path, preprocessed_path, skip_symbolic_shape=True)
    oq.quantize_static(preprocessed_path, quantized_path, Mock(),
                       calibrate_method=oq.CalibrationMethod.Percentile,
                       op_types_to_quantize=["Conv", "Mul", "Gemm"]) 

Urgency

Urgent, since it worked with previous versions of onnxruntime-gpu.

Platform

Windows

OS Version

10

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

CUDA 11.6

[xadupre]

PR #19591 fixes this issue. Am I allowed to use the model to write the unit test checking the bug never shows up again?

[onnxruntime-user]

Yes, you can use the model for the test. Thanks for the quick fix! Kind regards, Stefan Erlbeck Von: Xavier Dupré ***@***.***> Gesendet: Mittwoch, 21. Februar 2024 19:08 An: microsoft/onnxruntime ***@***.***> Cc: Stefan Erlbeck ***@***.***>; Author ***@***.***> Betreff: Re: [microsoft/onnxruntime] Static quantization crashes with "TypeError: iteration over a 0-d array" (Issue #19529) PR #19591<#19591> fixes this issue. Am I allowed to use the model to write the unit test checking the bug never shows up again? — Reply to this email directly, view it on GitHub<#19529 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BGFXI3TVZIXRWE6SVWQO6W3YUYZXNAVCNFSM6AAAAABDKHMLR2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJXGU3DSMRXGA>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>