[Build] build from source with cuda12.2 failed

[Gpwner]

Describe the issue

Hi,I have notice that 'Not officially supporting CUDA 12’.But I want to have a shot.here come a problem:
Compiling the CUDA compiler identification source file "CMakeCUDACompilerId.cu" failed.

Urgency

yes it is urgent

Target platform

windows11

Build script

.\build.bat --config RelWithDebInfo --build_wheel --compile_no_warning_as_error --skip_submodule_sync --use_cuda --cuda_home D:\\CUDA12.2 --cudnn_home D:\\CUDA12.2 --cuda_version=12.2 --nvcc_threads 1 --cmake_generator "Visual Studio 17 2022"

Error / output

-- ******** Summary ********
--   CMake version                     : 3.28.1
--   CMake command                     : D:/Program Files/CMake/bin/cmake.exe
--   System                            : Windows
--   C++ compiler                      : D:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe
--   C++ compiler version              : 19.38.33134.0
--   CXX flags                         : /guard:cf /DWIN32 /D_WINDOWS /MP /DWINAPI_FAMILY=100 /DWINVER=0x0A00 /D_WIN32_WINNT=0x0A00 /DNTDDI_VERSION=0x0A000000 /O2 /Ob1 /DNDEBUG /EHsc /EHsc /wd26812 -DEIGEN_HAS_C99_MATH -DCPUINFO_SUPPORTED /EHsc /wd26812
--   Build type                        : RelWithDebInfo
--   Compile definitions               : ORT_ENABLE_STREAM;EIGEN_MPL2_ONLY;EIGEN_HAS_CONSTEXPR;EIGEN_HAS_VARIADIC_TEMPLATES;EIGEN_HAS_CXX11_MATH;EIGEN_HAS_CXX11_ATOMIC;EIGEN_STRONG_INLINE=inline;__STDC_FORMAT_MACROS
--   CMAKE_PREFIX_PATH                 : D:\onnxruntime\\build\Windows/RelWithDebInfo/installed
--   CMAKE_INSTALL_PREFIX              : C:/Program Files/onnxruntime
--   CMAKE_MODULE_PATH                 : D:/onnxruntime/cmake/external
--
--   ONNX version                      : 1.15.0
--   ONNX NAMESPACE                    : onnx
--   ONNX_USE_LITE_PROTO               : ON
--   USE_PROTOBUF_SHARED_LIBS          : OFF
--   Protobuf_USE_STATIC_LIBS          : ON
--   ONNX_DISABLE_EXCEPTIONS           : OFF
--   ONNX_DISABLE_STATIC_REGISTRATION  : OFF
--   ONNX_WERROR                       : OFF
--   ONNX_BUILD_TESTS                  : OFF
--   ONNX_BUILD_BENCHMARKS             : OFF
--   ONNX_BUILD_SHARED_LIBS            :
--   BUILD_SHARED_LIBS                 : OFF
--
--   Protobuf compiler                 :
--   Protobuf includes                 :
--   Protobuf libraries                :
--   BUILD_ONNX_PYTHON                 : OFF
Finished fetching external dependencies
CMake Error at D:/Program Files/CMake/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:780 (message):
  Compiling the CUDA compiler identification source file
  "CMakeCUDACompilerId.cu" failed.

  Compiler:

  Build flags:
  /DWIN32;/D_WINDOWS;/DWINAPI_FAMILY=100;/DWINVER=0x0A00;/D_WIN32_WINNT=0x0A00;/DNTDDI_VERSION=0x0A000000;/DNDEBUG;-Xcompiler=
  /guard:cf /MP /O2 /Ob1

  Id flags: --keep;--keep-dir;tmp -v



  The output was:

  1

Visual Studio Version

Visual Studio community 2022 (17.8.5)

GCC / Compiler Version

(onnx) D:\onnxruntime>cmake --version cmake version 3.28.1 CMake suite maintained and supported by Kitware (kitware.com/cmake).

[Gpwner]

here is my nvcc info:

(onnx) D:\onnxruntime>nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Jun_13_19:42:34_Pacific_Daylight_Time_2023
Cuda compilation tools, release 12.2, V12.2.91
Build cuda_12.2.r12.2/compiler.32965470_0

(onnx) D:\onnxruntime>where nvcc
D:\CUDA12.2\bin\nvcc.exe

[tianleiwu]

Here is my build script:

IF {%VCToolsVersion%}=={} call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" amd64

set CudaToolkitDir=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2

.\build.bat --config Release --build_dir .\build --build_shared_lib --parallel 4 --nvcc_threads 1 --build_wheel ^
            --cmake_generator "Visual Studio 17 2022" ^
            --use_cuda --cuda_version 12.2 --cuda_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2" ^
            --cudnn_home d:\nvidia\cudnn-windows-x86_64-8.9.4.25_cuda12-archive ^
            --cmake_extra_defines CMAKE_CUDA_ARCHITECTURES=89 ^
            --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF  ^
            --skip_tests

You may try set CudaToolkitDir environment variable.

[snnn]

In the build folder this a folder named "CMakeFiles" which has more detailed logs that can tell us why compiling "CMakeCUDACompilerId.cu" failed . And, please provide the ONNX Runtime's source code version. If you were using the main branch, try to update it. yesterday I just merged an important change for the build system.

[Gpwner]

@tianleiwu @snnn I just reinstall cuda into the default location,and this error gone.thanks

[Gpwner]

I finally got my whl file,as 'onnxruntime_gpu-1.18.0-cp310-cp310-win_amd64.whl'.But when i install it to inference i got an error, can you help?

onnxruntime-gpu version:1.18.0   provider:['CUDAExecutionProvider', 'CPUExecutionProvider']
� [ 1 ; 3 1 m 2 0 2 4 - 0 1 - 3 1   1 5 : 4 3 : 1 4 . 1 4 0 3 3 5 2   [ E : o n n x r u n t i m e : ,   s e q u e n t i a l _ e x e c u t o r . c c : 5 1 4   o n n x r u n t i m e : : E x e c u t e K e r n e l ]   N o n - z e r o   s t a t u s   c o d e   r e t u r n e d   w h i l e   r u n n i n g   C o n c a t   n o d e .   N a m e : ' / C o n c a t '   S t a t u s   M e s s a g e :   C U D A   e r r o r   c u d a E r r o r N o K e r n e l I m a g e F o r D e v i c e : n o   k e r n e l   i m a g e   i s   a v a i l a b l e   f o r   e x e c u t i o n   o n   t h e   d e v i c e � [ m 
 Traceback (most recent call last):
  File "D:\PycharmProjects\onnx_infer\main.py", line 15, in <module>
    session.run(None, {input_name_1st: input_1st,
  File "C:\Users\xxx\.conda\envs\TEST_ONNX\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 220, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Concat node. Name:'/Concat' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device

[tianleiwu]

@Gpwner, that error means cuda architecture is not matched. You need change CMAKE_CUDA_ARCHITECTURES=89 according to your GPU Compute Capability (you can look up your GPU here: https://developer.nvidia.com/cuda-gpus). 89 is for compute Capability 8.9 (like RTX 4070).

[Gpwner]

@Gpwner, that error means cuda architecture is not matched. You need change CMAKE_CUDA_ARCHITECTURES=89 according to your GPU Compute Capability (you can look up your GPU here: https://developer.nvidia.com/cuda-gpus). 89 is for compute Capability 8.9 (like RTX 4070).

yeah,it works.thank you very much