Failure to build highway with Apple's clang 16

[fxcoudert]

With the latest git version of highway, on macOS 15 Beta, with Apple's system compiler (clang 16 beta), I configure with:

cmake -S . -B builddir -DBUILD_SHARED_LIBS=ON -DHWY_ENABLE_TESTS=OFF -DHWY_ENABLE_EXAMPLES=OFF

then build with

cmake --build builddir

and get the following crash:

fx@mini-m2 highway % cmake -S . -B builddir -DBUILD_SHARED_LIBS=ON -DHWY_ENABLE_TESTS=OFF -DHWY_ENABLE_EXAMPLES=OFF
CMake Deprecation Warning at CMakeLists.txt:28 (cmake_policy):
  The OLD behavior for policy CMP0111 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


-- The C compiler identification is AppleClang 16.0.0.16000026
-- The CXX compiler identification is AppleClang 16.0.0.16000026
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test ATOMICS_LOCK_FREE_INSTRUCTIONS
-- Performing Test ATOMICS_LOCK_FREE_INSTRUCTIONS - Success
-- Performing Test HWY_EMSCRIPTEN
-- Performing Test HWY_EMSCRIPTEN - Failed
-- Performing Test HWY_RISCV
-- Performing Test HWY_RISCV - Failed
-- Looking for sys/auxv.h
-- Looking for sys/auxv.h - not found
-- Looking for asm/hwcap.h
-- Looking for asm/hwcap.h - not found
-- Configuring done (1.3s)
-- Generating done (0.1s)
-- Build files have been written to: /tmp/highway/builddir
fx@mini-m2 highway % cmake --build builddir
[  2%] Building CXX object CMakeFiles/hwy.dir/hwy/abort.cc.o
[  5%] Building CXX object CMakeFiles/hwy.dir/hwy/aligned_allocator.cc.o
[  7%] Building CXX object CMakeFiles/hwy.dir/hwy/nanobenchmark.cc.o
[ 10%] Building CXX object CMakeFiles/hwy.dir/hwy/per_target.cc.o
[ 12%] Building CXX object CMakeFiles/hwy.dir/hwy/print.cc.o
[ 15%] Building CXX object CMakeFiles/hwy.dir/hwy/targets.cc.o
[ 17%] Building CXX object CMakeFiles/hwy.dir/hwy/timer.cc.o
[ 20%] Linking CXX shared library libhwy.dylib
[ 20%] Built target hwy
[ 22%] Building CXX object CMakeFiles/hwy_contrib.dir/hwy/contrib/sort/vqsort_128a.cc.o
c++: error: unable to execute command: Segmentation fault: 11
c++: error: clang frontend command failed due to signal (use -v to see invocation)
Apple clang version 16.0.0 (clang-1600.0.26.3)
Target: arm64-apple-darwin24.0.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
c++: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
c++: note: diagnostic msg: /var/folders/3s/c_rqj2651cq74hv7rv55dmh40000gn/T/vqsort_128a-4d5f40.cpp
c++: note: diagnostic msg: /var/folders/3s/c_rqj2651cq74hv7rv55dmh40000gn/T/vqsort_128a-4d5f40.sh
c++: note: diagnostic msg: Crash backtrace is located in
c++: note: diagnostic msg: /Users/fx/Library/Logs/DiagnosticReports/clang_<YYYY-MM-DD-HHMMSS>_<hostname>.crash
c++: note: diagnostic msg: (choose the .crash file that corresponds to your crash)
c++: note: diagnostic msg: 

********************
gmake[2]: *** [CMakeFiles/hwy_contrib.dir/build.make:76: CMakeFiles/hwy_contrib.dir/hwy/contrib/sort/vqsort_128a.cc.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:143: CMakeFiles/hwy_contrib.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

[fxcoudert]

I have filed the clang bug with Apple as FB14947714. A standalone reproducer is:

"/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" "-cc1" "-triple" "arm64-apple-macosx15.0.0" "-Wundef-prefix=TARGET_OS_" "-Wdeprecated-objc-isa-usage" "-Werror=deprecated-objc-isa-usage" "-Werror=implicit-function-declaration" "-emit-obj" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "vqsort_i16d.cc" "-mrelocation-model" "pic" "-pic-level" "2" "-fmerge-all-constants" "-mframe-pointer=non-leaf" "-mllvm" "-extra-vectorizer-passes" "-fno-strict-return" "-fmath-errno" "-ffp-contract=on" "-fno-rounding-math" "-fobjc-msgsend-selector-stubs" "-target-sdk-version=15.0" "-fvisibility-inlines-hidden-static-local-var" "-fno-modulemap-allow-subdirectory-search" "-target-cpu" "apple-m1" "-target-feature" "+v8.5a" "-target-feature" "+aes" "-target-feature" "+crc" "-target-feature" "+dotprod" "-target-feature" "+fp-armv8" "-target-feature" "+fp16fml" "-target-feature" "+lse" "-target-feature" "+ras" "-target-feature" "+rcpc" "-target-feature" "+rdm" "-target-feature" "+sha2" "-target-feature" "+sha3" "-target-feature" "+neon" "-target-feature" "+zcm" "-target-feature" "+zcz" "-target-feature" "+fullfp16" "-target-abi" "darwinpcs" "-debugger-tuning=lldb" "-target-linker-version" "1115.7.3" "-fcoverage-compilation-dir=/private/tmp/highway-20240902-88422-obt0s/builddir" "-skip-unused-modulemap-deps" "-sys-header-deps" "-D" "HWY_SHARED_DEFINE" "-D" "TOOLCHAIN_MISS_ASM_HWCAP_H" "-D" "TOOLCHAIN_MISS_SYS_AUXV_H" "-D" "hwy_contrib_EXPORTS" "-D" "NDEBUG" "-D" "__DATE__=\"redacted\"" "-D" "__TIMESTAMP__=\"redacted\"" "-D" "__TIME__=\"redacted\"" "-source-date-epoch" "1724771854" "-O3" "-Wno-builtin-macro-redefined" "-Wall" "-Wextra" "-Wconversion" "-Wsign-conversion" "-Wvla" "-Wnon-virtual-dtor" "-Wcast-align" "-Wfloat-overflow-conversion" "-Wfloat-zero-conversion" "-Wfor-loop-analysis" "-Wgnu-redeclared-enum" "-Winfinite-recursion" "-Wself-assign" "-Wstring-conversion" "-Wtautological-overlap-compare" "-Wthread-safety-analysis" "-Wundefined-func-template" "-Wc++2a-extensions" "-Wno-psabi" "-Wno-reorder-init-list" "-Wno-implicit-int-float-conversion" "-Wno-c99-designator" "-Wno-final-dtor-non-final-class" "-Wno-extra-semi-stmt" "-Wno-misleading-indentation" "-Wno-quoted-include-in-framework-header" "-Wno-implicit-fallthrough" "-Wno-enum-enum-conversion" "-Wno-enum-float-conversion" "-Wno-elaborated-enum-base" "-Wno-reserved-identifier" "-Wno-gnu-folding-constant" "-std=c++17" "-fdeprecated-macro" "-fdebug-compilation-dir=/private/tmp/highway-20240902-88422-obt0s/builddir" "-fdebug-prefix-map=/private/tmp/highway-20240902-88422-obt0s=." "-ferror-limit" "19" "-fvisibility=hidden" "-fvisibility-inlines-hidden" "-stack-protector" "1" "-fstack-check" "-mdarwin-stkchk-strong-link" "-fblocks" "-fencode-extended-block-signature" "-fregister-global-dtors-with-atexit" "-fgnuc-version=4.2.1" "-fno-cxx-modules" "-fmax-type-align=16" "-fcommon" "-fcolor-diagnostics" "-clang-vendor-feature=+disableNonDependentMemberExprInCurrentInstantiation" "-fno-odr-hash-protocols" "-clang-vendor-feature=+enableAggressiveVLAFolding" "-clang-vendor-feature=+revert09abecef7bbf" "-clang-vendor-feature=+thisNoAlignAttr" "-clang-vendor-feature=+thisNoNullAttr" "-clang-vendor-feature=+disableAtImportPrivateFrameworkInImplementationError" "-x" "c++" "vqsort_i16d-314a0c.cpp"

with the vqsort_i16d-314a0c.cpp file attached.
vqsort_i16d-314a0c.cpp.zip

[jan-wassenberg]

Thanks for filing the clang bug :) If you are not using VQSort, how about defining VQSORT_ENABLED to 0 as a workaround? (This requires a small patch which I'll send shortly.)

[fxcoudert]

@jan-wassenberg I'm a Homebrew maintainer, so I can't really presume what our users use or do not use. I'm not sure what triggers the crash, haven't been able to reduce it so far.

[jan-wassenberg]

Got it, thanks. I was able to repro on Godbolt. This looks like a catastrophic compiler bug, it seems unable to mangle its own definition of vaddq_u64:

0.	Program arguments: /opt/compiler-explorer/clang-17.0.1/bin/clang++ -gdwarf-4 -g -o /app/output.s -fno-verbose-asm -S -target aarch64-linux-gnu --gcc-toolchain=/opt/compiler-explorer/arm64/gcc-13.2.0/aarch64-unknown-linux-gnu --sysroot=/opt/compiler-explorer/arm64/gcc-13.2.0/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot -fcolor-diagnostics -fno-crash-diagnostics -std=c++17 -O3 -fcolor-diagnostics -x c++ -Xclang -triple -Xclang arm64-apple-macosx15.0.0 <source> -isystem/opt/compiler-explorer/libs/highway/trunk
1.	/opt/compiler-explorer/clang-17.0.1/lib/clang/17/include/arm_neon.h:2198:1 <Spelling=/opt/compiler-explorer/clang-17.0.1/lib/clang/17/include/arm_neon.h:454:14>: current parser token 'static'
2.	/opt/compiler-explorer/clang-17.0.1/lib/clang/17/include/arm_neon.h:2181:17: LLVM IR generation of declaration 'vaddq_u64'
3.	/opt/compiler-explorer/clang-17.0.1/lib/clang/17/include/arm_neon.h:2181:17: Mangling declaration 'vaddq_u64'
#0 0x0000000003349f18 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/compiler-explorer/clang-17.0.1/bin/clang+++0x3349f18)
#1 0x0000000003348044 llvm::sys::CleanupOnSignal(unsigned long) (/opt/compiler-explorer/clang-17.0.1/bin/clang+++0x3348044)
#2 0x000000000329c028 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
#3 0x0000763dbde42520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
#4 0x0000000000ac9492 (anonymous namespace)::CXXNameMangler::mangleExpression(clang::Expr const*, unsigned int, bool) (.cold) ItaniumMangle.cpp:0:0
clang++: error: clang frontend command failed with exit code 139 (use -v to see invocation)

It is caused by the -Xclang -triple -Xclang arm64-apple-macosx15.0.0 argument. Are we somehow able to convince CMake to instead go with -Xclang -triple -Xclang arm64-apple-unknown? That seems to work.

[Bo98]

The Godbolt error is a little misleading as it is using a Linux sysroot rather than the macOS SDK, but it does showcase it crashes nonetheless. The stack trace seen in the crash logs on actual macOS aligns with what was posted in llvm/llvm-project#97198 (comment) rather than being about mangling.

It's crashing in the arm64 SVE code. The reason it wasn't an issue before is because the SVE parts were never compiled before because the Apple Clang didn't have dynamic dispatch enabled (previously LLVM 15 based):

highway/hwy/detect_targets.h

Lines 621 to 623 in cce7182

	#if HWY_ARCH_ARM_A64 && HWY_OS_APPLE && \
	(HWY_COMPILER_GCC_ACTUAL || HWY_COMPILER_CLANG >= 1700)
	#define HWY_HAVE_RUNTIME_DISPATCH_APPLE 1

So previously it would only ever compile NEON:

Compiled HWY_TARGETS:   NEON
HWY_ATTAINABLE_TARGETS: NEON NEON_WITHOUT_AES EMU128
HWY_BASELINE_TARGETS:   NEON NEON_WITHOUT_AES EMU128
HWY_STATIC_TARGET:      NEON
HWY_BROKEN_TARGETS:     NEON_BF16
HWY_DISABLED_TARGETS:  
Current CPU supports:   NEON NEON_WITHOUT_AES EMU128 SCALAR

Apple Clang 16 now aligns with LLVM 17 so this code path is now enabled and thus SVE is now compiled. This now triggers the crashing code.

However, no Apple arm64 CPU supports SVE as far as I'm aware of and Clang seems to have always crashed when compiling arm64 SVE on macOS - see llvm/llvm-project#49751 where upstream states ARM SVE is not supported on macOS yet. Upstream also asked why SVE was being used here: llvm/llvm-project#97198 (comment).

I'd still call it a compiler bug - it shouldn't ever be crashing like that - but we can work around it.

The below seems to fix the crash issue for me:

diff --git a/hwy/detect_targets.h b/hwy/detect_targets.h
index a8d4a13f..e0ffb33a 100644
--- a/hwy/detect_targets.h
+++ b/hwy/detect_targets.h
@@ -223,8 +223,12 @@
 #endif
 
 // SVE[2] require recent clang or gcc versions.
+//
+// SVE is not supported on Apple arm64 CPUs and also crashes the compiler:
+// https://github.com/llvm/llvm-project/issues/97198
 #if (HWY_COMPILER_CLANG && HWY_COMPILER_CLANG < 1100) || \
-    (HWY_COMPILER_GCC_ACTUAL && HWY_COMPILER_GCC_ACTUAL < 1000)
+    (HWY_COMPILER_GCC_ACTUAL && HWY_COMPILER_GCC_ACTUAL < 1000) || \
+    (HWY_OS_APPLE && HWY_ARCH_ARM_A64)
 #define HWY_BROKEN_SVE (HWY_SVE | HWY_SVE2 | HWY_SVE_256 | HWY_SVE2_128)
 #else
 #define HWY_BROKEN_SVE 0

I can send a PR if something like that is OK. If ARM SVE is ever supported in the future a condition can be added with the minimum supported Clang version.

[jan-wassenberg]

Thanks, I'm back and agree with your analysis. Indeed M4 still doesn't have SVE, I thought it was mandatory in armv9 but they seem to have gone back on that.

We'd welcome such a pull request, and while at it we can change to #define HWY_BROKEN_SVE HWY_ALL_SVE which is more future-proof.