arm64: force -mcpu to be valid

[clementperon]

Description

When building with a cross-toolchain some CXX_FLAGS could be set to a specific CPU.

This leads to a CPU not compatible with some specific functions compiles with ARCH v8.2+bf16

Error is :
cc1: error: switch '-mcpu=XXXX' conflicts with '-march=armv8.2-a+i8mm' switch

Motivation and Context

As these functions are dispatched at runtime and as there is no -mno-cpu, force the -mcpu to be valid by forcing it to a valid combination of -march and -mcpu

[snnn]

According to https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/compiler-flags-across-architectures-march-mtune-and-mcpu, we should not need to use both "-march" and "-mcpu"

[clementperon]

@snnn according to your article we should avoid -march and only use -mcpu.

What would you say about only keeping -mcpu in this patch ? This would also fix my issue

[lida2003]

Hi, @clementperon, I have met this #23267 / v1.19.2 for Jetpack 5.1.4 L4T 35.6

Is this patch ok to build without "BFLOAT16" issue?

EDIT: I modified cmake/onnxruntime_mlas.cmake according to the patch, still got BFLOAT16 error.

-- Looking for reallocarray
-- Looking for reallocarray - found
-- Performing Test HAS_ARM64_BFLOAT16
-- Performing Test HAS_ARM64_BFLOAT16 - Failed
CMake Error at CMakeLists.txt:677 (message):
  The compiler doesn't support BFLOAT16!!!

EDIT2: Do I need extra flags in build command?

./build.sh --config Release --update --build --parallel --build_wheel \
--use_tensorrt --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu \
--tensorrt_home /usr/lib/aarch64-linux-gnu

EDIT3: difference between v1.19.2 and my onnx: 21117_for_v1.19.2.patch

[snnn]

@lida2003 , would you please consider upgrading your jetpack to 6.0(which has Ubuntu 22.04 and GCC 11)?

[lida2003]

@snnn We are focusing ROS/ubuntu20.04, Jetpack 5.1.4 is the up to date release for this. JP6 doesn't support this. And I think there are quite a lot of users are ON JP5 now.

EDIT: #21383 - "JetPack 6.0 is not an option for Xavier or Nano users!"

[snnn]

Even ubuntu20.04 itself will reach EOL in a few months. Sorry we do not have bandwidth to maintain a build system for ubuntu20.04 and GCC 9. You may fork this repo and patch it in your way.

[snnn]

@liqunfu , I am not familiar with mlas code. If you would like to take the changes, please help review the PR. Otherwise I will close it later.

[lida2003]

You may fork this repo and patch it in your way.

@snnn That's OK.

Currently the treads end up here, but the patch is not working. Is there any flags or other links/help available for me?

EDIT: what's the root cause for this? in #21099 , it seems "Some ARCH64 machine doesn't have support for dotprod or bf16". Is this mean that it's gcc issue on the current ubuntu 20.04 gcc 9.4.0?

Which version has fixed this, any link? Or is there an alternative way to get around gcc 9.4.0 and get onnx build?

$ gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

EDIT2: In #22837 #19760, still some gcc version & bf16 issue. If onnx is tested bf16 with gcc 11,12 and 13, is it possible to remove bf16 when gcc version is less than 11 ?

[snnn]

You may search the build flag in https://github.com/microsoft/onnxruntime/blame/main/cmake/onnxruntime_mlas.cmake

[snnn]

I believe it was added in #14538