[Delivery] Win ARM64 wheels + QNN

[khmyznikov]

Describe the feature request

Creating general issue to track ONNX DX on Windows ARM platform, particularly QNN.

Main problems:

• Lack of pre-build arm libs/binaries/wheels.

  The necessity of building from source even our own tools create huge obstacles and difficulties for DX. Launching basic SDK demos and examples like this requires a lot of time and mental resources. Days at best, instead of minutes or hours at worse.

• Necessity to mix/switch x64 and Arm environments to run basic demos.

  This is partially a result of previous problem. Qualcomm SDK and onnxruntime demos for QNN are not arm friendly. Some demos to the point of impossibility of execution on arm machine even with x64 emulation (see lack of AVX instructions emulation).

• Feels like a development kit rather than a development device.

  Due to the above issues, the DX with Python and ML feels like you are still using an experimental device and requires an additional x64 machine for development. Which is not the case for example for NodeJS/Web development, because everything is cross-platform or arm native already.

Opportunities to improve current state:

• Fix lack of python wheels for onnx toolkits and runtimes.
  • onnxruntime - @adrianlizarraga
  • onnxruntime-extensions - @wenbingl
  • onnx
• Make sure our demos are arm ready with no manual build of libraries.
  • onnxruntime-inference-examples
  • ... add more
• Make Olive toolkit available for win arm
  • Connect efforts of Win AI team and Olive team (Olive QNN optimizations)
• Invest to presence of 3rd party python libs on win arm
  • torch
  • numpy
  • ... add more
• Improve x64 emulator to support modern CPU instructions
  This can be a backup plan to cover some gaps can't be covered by ourself.

Describe scenario use case

The ability to use Windows ARM64 as main machine for ML development, not only as inference target.

[jywu-msft]

Thanks for filing these issues.
QNN EP nightly packages for nuget and python wheels is in progress and should be available shortly. +@adrianlizarraga

[khmyznikov]

QNN EP nightly packages for nuget and python wheels is in progres

Is the runtime-extensions will be included?

[jywu-msft]

QNN EP nightly packages for nuget and python wheels is in progres

Is the runtime-extensions will be included?

onnxruntime-extensions is a separate project/package. I see you've already filed an issue at microsoft/onnxruntime-extensions#624
+@wenbingl fyi

[jywu-msft]

see https://onnxruntime.ai/docs/execution-providers/QNN-ExecutionProvider.html#pre-built-packages for ort-qnn-nightly python package installation instructions

[jywu-msft]

pip install onnx runs cleanly on Win/ARM devices. with recent ONNX versions (e.g. ONNX 1.15)

[hmartinez82]

@jywu not here 😢 I can't for the life of me get it to work.
I'm on a ARM64 Windows 11 machine. It tries to build onnx and some of its deps locally and it always fails.

        randen_slow.cc
        randen_hwaes.cc
        utf8_range.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\utf8_range\Release\utf8_range.lib
        scoped_set_env.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\base\Release\absl_scoped_set_env.lib
      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.dir\Release\random_i.C13AF9DB.tlog\Lib-link.read.1.tlog. The system cannot find the path specified. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.vcxproj]



      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.dir\Release\random_i.C13AF9DB.tlog\Lib-link.write.1.tlog. The system cannot find the path specified. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.vcxproj]



        random_internal_seed_material.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\Release\absl_random_internal_seed_material.lib
        random_internal_randen_slow.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\Release\absl_random_internal_randen_slow.lib
        usage.cc
        libprotobuf-lite.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\Release\libprotobuf-lite.lib
        flags_usage.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\Release\absl_flags_usage.lib
      C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(541,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\flags_parse.vcxproj]
        Building Custom Rule C:/Users/herna/AppData/Local/Temp/pip-install-avijtn3e/onnx_9b6c206d4d2a4980900839fa2d1f9a2e/.setuptools-cmake-build/_deps/abseil-src/absl/flags/CMakeLists.txt
        parse.cc
        flags_parse.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\Release\absl_flags_parse.lib
      Traceback (most recent call last):
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 410, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 395, in _build_with_temp_dir
          self.run_setup()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 309, in <module>
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 200, in run_commands
          dist.run_commands()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\normal\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 132, in run
          self.run_command(cmd_name)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 246, in run
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 225, in run
        File "C:\Program Files\Python311\Lib\subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['C:\\Users\\herna\\AppData\\Local\\Temp\\pip-build-env-w6cyhan5\\overlay\\Scripts\\cmake.EXE', '--build', '.', '--config', 'Release', '--', '/maxcpucount:8']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for onnx
Failed to build onnx
ERROR: Could not build wheels for onnx, which is required to install pyproject.toml-based projects

This is with Python 3.11.9. Which Python version were you using?

[jywu-msft]

@jywu not here 😢 I can't for the life of me get it to work. I'm on a ARM64 Windows 11 machine. It tries to build onnx and some of its deps locally and it always fails.

        randen_slow.cc
        randen_hwaes.cc
        utf8_range.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\utf8_range\Release\utf8_range.lib
        scoped_set_env.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\base\Release\absl_scoped_set_env.lib
      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.dir\Release\random_i.C13AF9DB.tlog\Lib-link.read.1.tlog. The system cannot find the path specified. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.vcxproj]



      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.dir\Release\random_i.C13AF9DB.tlog\Lib-link.write.1.tlog. The system cannot find the path specified. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.vcxproj]



        random_internal_seed_material.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\Release\absl_random_internal_seed_material.lib
        random_internal_randen_slow.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\Release\absl_random_internal_randen_slow.lib
        usage.cc
        libprotobuf-lite.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\Release\libprotobuf-lite.lib
        flags_usage.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\Release\absl_flags_usage.lib
      C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(541,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\flags_parse.vcxproj]
        Building Custom Rule C:/Users/herna/AppData/Local/Temp/pip-install-avijtn3e/onnx_9b6c206d4d2a4980900839fa2d1f9a2e/.setuptools-cmake-build/_deps/abseil-src/absl/flags/CMakeLists.txt
        parse.cc
        flags_parse.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\Release\absl_flags_parse.lib
      Traceback (most recent call last):
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 410, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 395, in _build_with_temp_dir
          self.run_setup()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 309, in <module>
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 200, in run_commands
          dist.run_commands()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\normal\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 132, in run
          self.run_command(cmd_name)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 246, in run
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 225, in run
        File "C:\Program Files\Python311\Lib\subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['C:\\Users\\herna\\AppData\\Local\\Temp\\pip-build-env-w6cyhan5\\overlay\\Scripts\\cmake.EXE', '--build', '.', '--config', 'Release', '--', '/maxcpucount:8']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for onnx
Failed to build onnx
ERROR: Could not build wheels for onnx, which is required to install pyproject.toml-based projects

This is with Python 3.11.9. Which Python version were you using?

@jywu not here 😢 I can't for the life of me get it to work. I'm on a ARM64 Windows 11 machine. It tries to build onnx and some of its deps locally and it always fails.

        randen_slow.cc
        randen_hwaes.cc
        utf8_range.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\utf8_range\Release\utf8_range.lib
        scoped_set_env.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\base\Release\absl_scoped_set_env.lib
      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.dir\Release\random_i.C13AF9DB.tlog\Lib-link.read.1.tlog. The system cannot find the path specified. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.vcxproj]



      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.dir\Release\random_i.C13AF9DB.tlog\Lib-link.write.1.tlog. The system cannot find the path specified. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\random_internal_randen_hwaes_impl.vcxproj]



        random_internal_seed_material.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\Release\absl_random_internal_seed_material.lib
        random_internal_randen_slow.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\random\Release\absl_random_internal_randen_slow.lib
        usage.cc
        libprotobuf-lite.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\Release\libprotobuf-lite.lib
        flags_usage.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\Release\absl_flags_usage.lib
      C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(541,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\flags_parse.vcxproj]
        Building Custom Rule C:/Users/herna/AppData/Local/Temp/pip-install-avijtn3e/onnx_9b6c206d4d2a4980900839fa2d1f9a2e/.setuptools-cmake-build/_deps/abseil-src/absl/flags/CMakeLists.txt
        parse.cc
        flags_parse.vcxproj -> C:\Users\herna\AppData\Local\Temp\pip-install-avijtn3e\onnx_9b6c206d4d2a4980900839fa2d1f9a2e\.setuptools-cmake-build\_deps\protobuf-build\third_party\abseil-cpp\absl\flags\Release\absl_flags_parse.lib
      Traceback (most recent call last):
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Roaming\Python\Python311-arm64\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 410, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 395, in _build_with_temp_dir
          self.run_setup()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 309, in <module>
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 200, in run_commands
          dist.run_commands()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\normal\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 132, in run
          self.run_command(cmd_name)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 246, in run
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\dist.py", line 967, in run_command
          super().run_command(command)
        File "C:\Users\herna\AppData\Local\Temp\pip-build-env-w6cyhan5\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 225, in run
        File "C:\Program Files\Python311\Lib\subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['C:\\Users\\herna\\AppData\\Local\\Temp\\pip-build-env-w6cyhan5\\overlay\\Scripts\\cmake.EXE', '--build', '.', '--config', 'Release', '--', '/maxcpucount:8']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for onnx
Failed to build onnx
ERROR: Could not build wheels for onnx, which is required to install pyproject.toml-based projects

This is with Python 3.11.9. Which Python version were you using?

you're right. this was previously working. something must have broken with onnx build on arm64. we will need to follow up and report to onnx project

[hmartinez82]

@jywu I'm not a Python dev.. Do you think that this could be a build issue. For instance, is it possible to try to build it with Ninja instead of MSBuild and see if it works?

[jywu-msft]

@jywu I'm not a Python dev.. Do you think that this could be a build issue. For instance, is it possible to try to build it with Ninja instead of MSBuild and see if it works?

yes, it's a build issue. the root problem is that onnx project doesn't publish python packages for win/arm64 platform to pypi.
so when you do a pip install on that platform, it tries to build from source. previously the build was succeeding but something is causing it to fail now.
I will file an issue against the onnx project https://github.com/onnx/onnx

[ashumish-QCOM]

Hi @khmyznikov

Qualcomm NPU (QNN-EP) with ONNX Runtime should be feasible. However, the lack of pre-built ARM64 libraries and binaries can indeed create significant obstacles. Ensuring that your environment paths are correctly set and that you have the necessary dependencies installed is crucial.

For those looking to integrate QNN-EP, make sure to follow the installation instructions provided in the ONNX Runtime documentation. This setup has been effective for other models available from the Qualcomm AI Hub.

Thankyou