Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] feat: more precise qbinary handling in tract core with unaligned entry #1333

Merged

Conversation

JulienBalianSonos
Copy link
Contributor

@JulienBalianSonos JulienBalianSonos commented Feb 9, 2024

Change ways binary ops are done for math with quantized tensors, by allowing unaligned qparams in input/output tensors.

This change allows increasing precision of quantization and reduce casting/alignment cost, but we now need to handle the qparams of output tensor as function input.

We need to handle all ops implied by bin_to_super_type macro already supporting QU8, that may use unaligned qparams quantization.

  • elementwise mul
  • div

in next cases we directly apply dequant to f32 and requant to targeted qtype for now

  • add
  • sub
  • min
  • max (here we set a small optim to make it slightly faster for ReLU case)

we propose to leave aside from this PR: rem, and all "bool" or "bit" operations

@JulienBalianSonos JulienBalianSonos force-pushed the fix/elmwise-mul-quant-algined-zp-scale branch from e02b862 to 2929ae7 Compare February 21, 2024 15:10
@JulienBalianSonos JulienBalianSonos changed the title [WIP] feat: more precise qmul handling in tract core with unaligned entry [RFC] feat: more precise qmul handling in tract core with unaligned entry Feb 29, 2024
@JulienBalianSonos JulienBalianSonos changed the title [RFC] feat: more precise qmul handling in tract core with unaligned entry [RFC] feat: more precise qbinary handling in tract core with unaligned entry Feb 29, 2024
core/src/ops/binary.rs Outdated Show resolved Hide resolved
core/src/ops/math/mod.rs Outdated Show resolved Hide resolved
core/src/ops/math/mod.rs Outdated Show resolved Hide resolved
nnef/src/registry.rs Outdated Show resolved Hide resolved
test-rt/suite-unit/src/q_binary.rs Outdated Show resolved Hide resolved
@kali kali merged commit 99ebf61 into sonos:main Mar 5, 2024
44 of 45 checks passed
@JulienBalianSonos JulienBalianSonos deleted the fix/elmwise-mul-quant-algined-zp-scale branch March 5, 2024 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants