[Build] Build onnxruntime 1.15.1 fail

[GilbertPan97]

Describe the issue

I am in the process of attempting to compile onnxruntime 1.15.1 on my device, but I have encountered an error while compiling the provider. It's worth mentioning that I successfully compiled onnxruntime 1.12 on the same device in the past without any issues.

I sincerely appreciate your time and assistance with this matter. Your help means a lot to me, and I am grateful for any guidance or insights you can provide. Thank you!

Urgency

Target platform

Platform: Ubuntu 20.04.4 LTS; Processor: ARMv8 (aarch64); GPU: NVIDIA Tegra Orin (Jetson)

Build script

The compilation command used is as follows:

./build.sh --config Release --update --build --parallel --build_wheel --use_cuda --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu --skip_submodule_sync

Error / output

[ 59%] Built target onnxruntime_providers
[ 59%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/causal/fmha_v2_fp16_Causal_128_32_sm70.cubin.cc.o
[ 59%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/causal/fmha_v2_fp16_Causal_128_32_sm75.cubin.cc.o
[ 59%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/causal/fmha_v2_fp16_Causal_128_32_sm80.cubin.cc.o
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc: In member function ‘onnxruntime::common::Status onnxruntime::contrib::cuda::Attention<T>::ComputeInternal(onnxruntime::OpKernelContext*) const’:
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc:167:3: error: there are no arguments to ‘ORT_UNUSED_VARIABLE’ that depend on a template parameter, so a declaration of ‘ORT_UNUSED_VARIABLE’ must be available [-fpermissive]
  167 |   ORT_UNUSED_VARIABLE(is_mask_1d_key_seq_len_start);
      |   ^~~~~~~~~~~~~~~~~~~
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc:167:3: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
[ 59%] Building CXX object CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/causal/fmha_v2_fp16_Causal_128_32_sm86.cubin.cc.o
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc: In instantiation of ‘onnxruntime::common::Status onnxruntime::contrib::cuda::Attention<T>::ComputeInternal(onnxruntime::OpKernelContext*) const [with T = onnxruntime::MLFloat16]’:
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.h:21:10:   required from here
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc:167:22: error: ‘ORT_UNUSED_VARIABLE’ was not declared in this scope; did you mean ‘HAS_UNUSED_VARIABLE’?
  167 |   ORT_UNUSED_VARIABLE(is_mask_1d_key_seq_len_start);
      |   ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |   HAS_UNUSED_VARIABLE
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc: In instantiation of ‘onnxruntime::common::Status onnxruntime::contrib::cuda::Attention<T>::ComputeInternal(onnxruntime::OpKernelContext*) const [with T = float]’:
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.h:21:10:   required from here
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc:167:22: error: ‘ORT_UNUSED_VARIABLE’ was not declared in this scope; did you mean ‘HAS_UNUSED_VARIABLE’?
  167 |   ORT_UNUSED_VARIABLE(is_mask_1d_key_seq_len_start);
      |   ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |   HAS_UNUSED_VARIABLE
make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/build.make:2084: CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/attention.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 59%] Built target onnx_test_runner
[ 59%] Built target onnxruntime_perf_test
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/multihead_attention.cc: In instantiation of ‘onnxruntime::common::Status onnxruntime::contrib::cuda::MultiHeadAttention<T>::ComputeInternal(onnxruntime::OpKernelContext*) const [with T = onnxruntime::MLFloat16]’:
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/multihead_attention.h:22:10:   required from here
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/multihead_attention.cc:120:8: error: unused variable ‘is_mask_1d_key_seq_len_start’ [-Werror=unused-variable]
  120 |   bool is_mask_1d_key_seq_len_start = parameters.mask_type == AttentionMaskType::MASK_1D_KEY_SEQ_LEN_START;
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/multihead_attention.cc: In instantiation of ‘onnxruntime::common::Status onnxruntime::contrib::cuda::MultiHeadAttention<T>::ComputeInternal(onnxruntime::OpKernelContext*) const [with T = float]’:
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/multihead_attention.h:22:10:   required from here
/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/multihead_attention.cc:120:8: error: unused variable ‘is_mask_1d_key_seq_len_start’ [-Werror=unused-variable]
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/build.make:2182: CMakeFiles/onnxruntime_providers_cuda.dir/home/nvidia/Documents/panjiabin/onnxruntime/onnxruntime/contrib_ops/cuda/bert/multihead_attention.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1933: CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
Traceback (most recent call last):
  File "/home/nvidia/Documents/panjiabin/onnxruntime/tools/ci_build/build.py", line 2599, in <module>
    sys.exit(main())
  File "/home/nvidia/Documents/panjiabin/onnxruntime/tools/ci_build/build.py", line 2495, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "/home/nvidia/Documents/panjiabin/onnxruntime/tools/ci_build/build.py", line 1434, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/home/nvidia/Documents/panjiabin/onnxruntime/tools/ci_build/build.py", line 781, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/nvidia/Documents/panjiabin/onnxruntime/tools/python/util/run.py", line 49, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/cmake-3.26.4-linux-aarch64/bin/cmake', '--build', '/home/nvidia/Documents/panjiabin/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.

Visual Studio Version

No response

GCC / Compiler Version

No response

[snnn]

Please try the latest version: 1.16.0, and let me know if the problem still exists.

[snnn]

We no longer make patches for 1.15.x

[tianleiwu]

@GilbertPan97, if you would like to patch by yourself, replace ORT_UNUSED_VARIABLE by ORT_UNUSED_PARAMETER.
It was fixed in 1.16 or main branch.

[GilbertPan97]

@GilbertPan97, if you would like to patch by yourself, replace ORT_UNUSED_VARIABLE by ORT_UNUSED_PARAMETER. It was fixed in 1.16 or main branch.

Thanks, I'm trying to replace ORT with version 1.16.0

[GilbertPan97]

Please try the latest version: 1.16.0, and let me know if the problem still exists.

I am attempting to use onnxruntime 1.16.0, which can be successfully compiled on my Orin device, but when I call it within my code for compilation, I receive the following message:

The given version [16] is not supported; only versions 1 to 12 are supported in this build.

Subsequently, the code stops at the point of creating the environment, as shown below:

inline Env::Env(OrtLoggingLevel logging_level, _In_const_char* logid) {
  ThrowOnError(GetApi().CreateEnv(logging_level, logid, &p_));
  ......
}

This is quite perplexing because I encountered no issues when executing the same code with ort1.16.0 on my x86 device.

Additionally, I have noticed that many header files are missing from the packages (gpu version) on the onnxruntime release page.

[GilbertPan97]

I am attempting to use onnxruntime 1.16.0, which can be successfully compiled on my Orin device, but when I call it within my code for compilation, I receive the following message:

The given version [16] is not supported; only versions 1 to 12 are supported in this build.

It's my mistake; I didn't notice that the program called the wrong library, which is installed in the system