Failing to link on Windows with RTools44

[bgoodri]

Summary:

The rstanarm package does not link correctly on the next R for Windows.

Description:

It is some combination of LTO and RTools44 but possibly also csr_matrix_times_vector. @SteveBronder ?

Reproducible Steps:

Try to install rstanarm with the RTools44 toolchain and you get at the end:

g++  -std=gnu++17 -I"D:/RCompile/recent/R/include" -DNDEBUG -I"../inst/include" -I"D:/RCompile/CRANpkg/lib/4.4/StanHeaders/include/src" -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG -DRCPP_PARALLEL_USE_TBB=1 -DUSE_STANC3 -D_HAS_AUTO_PTR_ETC=0 -I'D:/RCompile/CRANpkg/lib/4.4/StanHeaders/include' -I'D:/RCompile/CRANpkg/lib/4.4/rstan/include' -I'D:/RCompile/CRANpkg/lib/4.4/BH/include' -I'D:/RCompile/CRANpkg/lib/4.4/Rcpp/include' -I'D:/RCompile/CRANpkg/lib/4.4/RcppEigen/include' -I'D:/RCompile/CRANpkg/lib/4.4/RcppParallel/include'   -I"d:/rtools44/x86_64-w64-mingw32.static.posix/include"  -DRCPP_PARALLEL_USE_TBB=1 -I"D:/RCompile/CRANpkg/lib/4.4/RcppParallel/include" -D_REENTRANT -DSTAN_THREADS   -flto=jobserver   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c init.cpp -o init.o
g++ -shared -s -static-libgcc -o rstanarm.dll rstanarm-win.def stan_files/bernoulli.o stan_files/binomial.o stan_files/continuous.o stan_files/count.o stan_files/jm.o stan_files/lm.o stan_files/mvmer.o stan_files/polr.o init.o -LD:/RCompile/CRANpkg/lib/4.4/RcppParallel/lib/x64 -ltbb -ltbbmalloc -LD:/RCompile/CRANpkg/lib/4.4/RcppParallel/lib/x64 -Wl,-rpath,D:/RCompile/CRANpkg/lib/4.4/RcppParallel/lib/x64 -ltbb -ltbbmalloc -Wl,--allow-multiple-definition -Ld:/rtools44/x86_64-w64-mingw32.static.posix/lib/x64 -Ld:/rtools44/x86_64-w64-mingw32.static.posix/lib -LD:/RCompile/recent/R/bin/x64 -lR
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZN4stan4math21reverse_pass_callbackIZNS0_23csr_matrix_times_vectorIN5Eigen3MapINS3_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS3_6StrideILi0ELi0EEEEENS5_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESL_RKT0_EUlvE2_EEvOSE_: symbol wrong type (4 vs 3)
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZN4stan4math8internal26reverse_pass_callback_variIZNS0_23csr_matrix_times_vectorIN5Eigen3MapINS4_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS4_6StrideILi0ELi0EEEEENS6_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESM_RKT0_EUlvE2_E16set_zero_adjointEv: symbol wrong type (4 vs 3)
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZN4stan4math8internal26reverse_pass_callback_variIZNS0_23csr_matrix_times_vectorIN5Eigen3MapINS4_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS4_6StrideILi0ELi0EEEEENS6_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESM_RKT0_EUlvE2_E5chainEv: symbol wrong type (4 vs 3)
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZN4stan4math8internal26reverse_pass_callback_variIZNS0_23csr_matrix_times_vectorIN5Eigen3MapINS4_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS4_6StrideILi0ELi0EEEEENS6_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESM_RKT0_EUlvE2_EC1EOSQ_: symbol wrong type (4 vs 3)
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZN4stan4math8internal26reverse_pass_callback_variIZNS0_23csr_matrix_times_vectorIN5Eigen3MapINS4_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS4_6StrideILi0ELi0EEEEENS6_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESM_RKT0_EUlvE3_E5chainEv: symbol wrong type (4 vs 3)
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZZN4stan4math23csr_matrix_times_vectorIN5Eigen3MapINS2_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS2_6StrideILi0ELi0EEEEENS4_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESK_RKT0_ENUlvE2_C1EOSO_: symbol wrong type (4 vs 3)
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZZN4stan4math23csr_matrix_times_vectorIN5Eigen3MapINS2_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS2_6StrideILi0ELi0EEEEENS4_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESK_RKT0_ENUlvE2_clEv: symbol wrong type (4 vs 3)
D:\rtools44\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot export _ZZN4stan4math23csr_matrix_times_vectorIN5Eigen3MapINS2_6MatrixIdLin1ELi1ELi0ELin1ELi1EEELi0ENS2_6StrideILi0ELi0EEEEENS4_INS0_9var_valueIdvEELin1ELi1ELi0ELin1ELi1EEELPv0EEEDaiiRKT_RKSt6vectorIiSaIiEESK_RKT0_ENUlvE3_clEv: symbol wrong type (4 vs 3)
collect2.exe: error: ld returned 1 exit status

RStanARM Version:

2.32.1

R Version:

4.4.0 alpha (2024-03-31 r86238 ucrt)

Operating System:

Windows only.

[bgoodri]

I can replicate this linker error on the Windows machine in my office.

[SteveBronder]

Are you able to run the above with the flag -Wno-ignored-attributes so we can see the other errors and warnings?

Would CRAN allow you to add the flag -fno-use-linker-plugin?

• https://stackoverflow.com/a/30465491

[bgoodri]

CRAN does not allow suppressing compiler warnings in released packages, but I can do it for a test build. See the installation log here, which was run with -Wno-ignored-attributes -Wno-sign-compare -Wno-unused-but-set-variable -Wno-deprecated-declarations -Wno-nonnull -Wno-parentheses -Wno-unused-variable -Wno-misleading-indentation in CXX17FLAGS and -fno-use-linker-plugin in PKG_LIBS. The same linker error persists.

[bgoodri]

Inlining the two internal namespaced functions used by csr_matrix_times_vector in StanHeaders did not fix the linker error on Windows for rstanarm.

[WardBrian]

I locally isolated the problem to be the 3 lambdas in csr_matrix_times_vector's reverse pass. We think the issue might be that the mangled names of the anonymous types being generated is too long (they are all conspicuously longer than 256 characters, and Windows has all sorts of weird max length conditions)

@SteveBronder is still thinking about solutions

[WardBrian]

@bgoodri I'm still reviewing it, but Steve's code changes in stan-dev/math#3048 should resolve this (I tested an earlier version of them with my own Windows machine)

Unfortunately they rely on some other changes in the Math lib beyond where SH is, namely stan-dev/math#2971, so it isn't just a one-file cherry pick

[bgoodri]

Merging multiple PRs into StanHeaders is fine as long as they are mostly additional to StanHeaders 2.32.x rather than backwards uncompatible. The main thing is that we only have a week before the development R becomes the stable R and may need to beat that deadline by a few days in order to have a binary rstanarm built for Windows when the release happens.

[bgoodri]

Did someone have to do something else (besides merging those two PRs) to get it to link on Windows? I am still getting the same error even after putting those two PRs into a provisional StanHeaders.

[WardBrian]

I did not. The last time I had access to a Windows box to test I was using an older version of the patch than the one that got merged -- it had the same signatures for the lambdas, but the math inside was wrong at the time. We were assuming (perhaps too optimistically?) that this wouldn't affect linking...

[bgoodri]

Some of the mangled names are longer than 256 characters, although it still is complaining about some that have shorter mangled names than that.

[WardBrian]

This is definitely still happening, and still goes away if you comment out the reverse_pass_callback lambdas.

[WardBrian]

@SteveBronder how difficult would it be to avoid using lambdas at all in the csr reverse pass?

[SteveBronder]

Not hard. I can make a PR for that tmrw

[WardBrian]

@bgoodri was this resolved by the latest Math changes?

I've still failed to re-create this with a simpler example to try to report upstream

[bgoodri]

Yes