[Mobile] need vulkan ep support

[WangHHY19931001]

Describe the issue

in qualcomm device select app ndk abiFilters armeabi-v7a, nnapi just only use cpu.

after google some, i fund qnn sdk not support armeabi-v7a, just only support arm64-v8a, maybe qualcomm nnapi not support npu for armeabi-v7a, so need vulkan ep to use gpu

To reproduce

build onnxruntime android with nnapi, create app, set ndk abiFilters armeabi-v7a, and use c++ call onnxruntime to run onnx

Urgency

No response

Platform

Android

OS Version

14

ONNX Runtime Installation

Built from Source

Compiler Version (if 'Built from Source')

26.3.11579264

Package Name (if 'Released Package')

None

ONNX Runtime Version or Commit ID

c4fb724

ONNX Runtime API

C++/C

Architecture

ARM32

Execution Provider

NNAPI

Execution Provider Library Version

No response

[wejoncy]

There is a webgpu EP you might be insterested in

[Red-Eyed]

There is a webgpu EP you might be insterested in

Is there a way to use webgpu in Android native (C++) library?

[skottmckay]

WebGPU EP uses the Dawn native library, which converts WGSL to Vulkan on Android. https://github.com/google/dawn/

The EP is new and only supports a limited number of operators currently, but those will be increased.

Not sure how well it supports 32-bit ARM though as most mobile phones are 64-bit ARM.

FWIW NNAPI is deprecated, and always had unpredictable performance.

[ashumish-QCOM]

Hi @WangHHY19931001,

I checked on this issue regarding the need for Vulkan EP support on Qualcomm devices. It appears that the QNN SDK does not support armeabi-v7a, only arm64-v8a. This might be why NNAPI is only using the CPU for armeabi-v7a.

For using the GPU, you might want to explore the WebGPU Execution Provider (EP), which uses the Dawn native library to convert WGSL to Vulkan on Android. However, please note that the WebGPU EP is relatively new and currently supports a limited number of operators, though this is expected to increase over time. Additionally, most mobile phones are 64-bit ARM, so support for 32-bit ARM might be limited.

FWIW, NNAPI is deprecated and has always had unpredictable performance.

Hope this helps!

[WangHHY19931001]

Hi @WangHHY19931001,

I checked on this issue regarding the need for Vulkan EP support on Qualcomm devices. It appears that the QNN SDK does not support armeabi-v7a, only arm64-v8a. This might be why NNAPI is only using the CPU for armeabi-v7a.

For using the GPU, you might want to explore the WebGPU Execution Provider (EP), which uses the Dawn native library to convert WGSL to Vulkan on Android. However, please note that the WebGPU EP is relatively new and currently supports a limited number of operators, though this is expected to increase over time. Additionally, most mobile phones are 64-bit ARM, so support for 32-bit ARM might be limited.

FWIW, NNAPI is deprecated and has always had unpredictable performance.

Hope this helps!

need way to build android c++ version

[Red-Eyed]

Hi @WangHHY19931001,
I checked on this issue regarding the need for Vulkan EP support on Qualcomm devices. It appears that the QNN SDK does not support armeabi-v7a, only arm64-v8a. This might be why NNAPI is only using the CPU for armeabi-v7a.
For using the GPU, you might want to explore the WebGPU Execution Provider (EP), which uses the Dawn native library to convert WGSL to Vulkan on Android. However, please note that the WebGPU EP is relatively new and currently supports a limited number of operators, though this is expected to increase over time. Additionally, most mobile phones are 64-bit ARM, so support for 32-bit ARM might be limited.
FWIW, NNAPI is deprecated and has always had unpredictable performance.
Hope this helps!

need way to build android c++ version

I haven't tried it yet, but there is option --use_webgpu in the build script.
I hope it will work with Android
https://github.com/microsoft/onnxruntime/blob/main/tools/ci_build/build.py#L579

[WangHHY19931001]

Hi @WangHHY19931001,
I checked on this issue regarding the need for Vulkan EP support on Qualcomm devices. It appears that the QNN SDK does not support armeabi-v7a, only arm64-v8a. This might be why NNAPI is only using the CPU for armeabi-v7a.
For using the GPU, you might want to explore the WebGPU Execution Provider (EP), which uses the Dawn native library to convert WGSL to Vulkan on Android. However, please note that the WebGPU EP is relatively new and currently supports a limited number of operators, though this is expected to increase over time. Additionally, most mobile phones are 64-bit ARM, so support for 32-bit ARM might be limited.
FWIW, NNAPI is deprecated and has always had unpredictable performance.
Hope this helps!

need way to build android c++ version

I haven't tried it yet, but there is option --use_webgpu in the build script. I hope it will work with Android https://github.com/microsoft/onnxruntime/blob/main/tools/ci_build/build.py#L579

onnxruntime\include\onnxruntime\core\providers\webgpu\webgpu_provider_factory.h is empty header

[Red-Eyed]

I found webgpu_provider_factory_creator.h
https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/webgpu/webgpu_provider_factory_creator.h

And there is even test for webgpu (only session creation)
https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/webgpu/external_dawn/main.cc#L48

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.