Compilation Failure on Arch Linux & distcc #145

bronze51 · 2020-06-07T19:14:04Z

I just started using distcc and successfully compiled "hello world", so I thought I move on to bigger things. The build for "vc4c-git" fails and I think it is due to phtread:

Using distcc to compile VC4C. The server is:

uname -a
Linux heimdallr 5.4.42-1-ARCH #1 SMP PREEMPT Tue May 26 01:48:52 UTC 2020 armv7l GNU/Linux

The host is:

uname -a
Linux walhalla 5.6.15-arch1-1 #1 SMP PREEMPT Wed, 27 May 2020 23:42:26 +0000 x86_64 GNU/Linux

Contents of "~/.cache/yay/vc4c-git/src/VC4C-master/build/CMakeFiles/CMakeError.log"

Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: ~/.cache/yay/vc4c-git/src/VC4C-master/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_fcdf3/fast && /usr/bin/make  -f CMakeFiles/cmTC_fcdf3.dir/build.make CMakeFiles/
cmTC_fcdf3.dir/build
make[1]: Entering directory '~/.cache/yay/vc4c-git/src/VC4C-master/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_fcdf3.dir/src.c.o
/usr/lib/distcc/bin/cc   -D_FORTIFY_SOURCE=2 -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector
-strong -fno-plt -DCMAKE_HAVE_LIBC_PTHREAD   -o CMakeFiles/cmTC_fcdf3.dir/src.c.o   -c ~/.cache/yay/vc4c-gi
t/src/VC4C-master/build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_fcdf3
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_fcdf3.dir/link.txt --verbose=1
/usr/lib/distcc/bin/cc -D_FORTIFY_SOURCE=2 -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector-s
trong -fno-plt -DCMAKE_HAVE_LIBC_PTHREAD  -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now  -rdynamic CMakeFiles/cmTC_f
cdf3.dir/src.c.o  -o cmTC_fcdf3
/usr/bin/ld: CMakeFiles/cmTC_fcdf3.dir/src.c.o: in function `main':
src.c:(.text.startup+0x38): undefined reference to `pthread_create'
/usr/bin/ld: src.c:(.text.startup+0x40): undefined reference to `pthread_detach'
/usr/bin/ld: src.c:(.text.startup+0x48): undefined reference to `pthread_cancel'
/usr/bin/ld: src.c:(.text.startup+0x54): undefined reference to `pthread_join'
collect2: error: ld returned 1 exit status

Complaints of "yay -S vc4c-git"

-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/lib/distcc/bin/cc
-- Check for working C compiler: /usr/lib/distcc/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/lib/distcc/bin/c++
-- Check for working CXX compiler: /usr/lib/distcc/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- build deb package...
-- Configuring done
-- Generating done
-- Build files have been written to: ~/.cache/yay/vc4c-git/src/VC4CLStdLib-master/build

...

make[1]: *** [CMakeFiles/Makefile2:1957: src/CMakeFiles/VC4CC.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Linking CXX shared library libcpptest-lite.so
[100%] Built target cpptest-lite
[ 42%] Built target cpptest-lite-project-build
make: *** [Makefile:183: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
Error making: vc4c-git

The text was updated successfully, but these errors were encountered:

doe300 · 2020-06-07T20:08:53Z

Does it work if you add -pthread to the list of libraries to link against in https://github.com/doe300/VC4C/blob/master/src/CMakeLists.txt#L74 ?

bronze51 · 2020-06-07T20:38:46Z

When I download the package form the AUR again to start the build using "yay", that file is overwritten and my addition is lost. I need to research how to tweak the build process or how to start it without overwritting the file ...

doe300 · 2020-06-08T05:22:27Z

Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:

Looking at this error, it looks like it fails to detect pthread in the following lines of the same CMake file.

Do you have the libpthread.so on your system? Is it located in your PATH or LIBRAY_PATH?

k8ieone · 2020-06-08T07:32:05Z

Hi, I own the Arch package and I pretty much gave up on it. If anyone wants I can transfer the ownership.

bronze51 · 2020-06-10T02:55:26Z

After unpacking the arch git package with makepkg --nobuild -s.
I tried to turn off MULTI_THREADED to bypass the issue with pthread.
I then run makepkg --noextract -i but now/still I hit:

distcc[10542] ERROR: compile ~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp on 192.168.1.190 failed
distcc[10542] (dcc_build_somewhere) Warning: remote compilation of '~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp' failed, retrying locally
distcc[10542] Warning: failed to distribute ~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp to 192.168.1.190, running locally instead

I turned off MULTI_THREADED in

./src/VC4C-master/build/src/CMakeFiles/VC4CC.dir/flags.make
./src/VC4C-master/build/src/CMakeFiles/VC4CC.dir/DependInfo.cmake
./src/VC4C-master/build/src/CMakeFiles/VC4C.dir/flags.make
./src/VC4C-master/build/src/CMakeFiles/VC4C.dir/DependInfo.cmake
./src/VC4C-master/CMakeLists.txt

bronze51 · 2020-06-10T03:00:42Z

libpthread.so is found in \usr\lib but not in PATH.

doe300 · 2020-06-10T05:06:54Z

I don't really know what is wrong, so all I can do is suggest a few things to test out:

As a work-around, you could try this: Could NOT find Threads (missing: Threads_FOUND) alicevision/geogram#2 (comment)
Or you could try setting these variables at the beginning of the main CMakeLists.txt (source)

set(THREADS_PREFER_PTHREAD_FLAG TRUE)
set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
Also, do you set some custom C/C++ build flags? E.g. via CFLAGS?

bronze51 · 2020-06-20T16:41:00Z

I was missing the following dependencies:
(Arch package names) "llvm", "opencl-headers" and "ocl-icd"

My next error is:

[  3%] Built target objvc4dis
[  8%] Built target objvc4asm
[ 13%] Built target objvc4base
[ 15%] Built target vc4dis
[ 17%] Built target vc4asm
[ 17%] Built target libvc4asm-static
[ 18%] Built target libvc4asm-shared
[ 18%] Performing update step for 'cpplog-project'
[ 20%] Performing configure step for 'cpplog-project'
-- Configuring done
-- Generating done
-- Build files have been written to: ~/dev/VC4C/build/cpplog/src/cpplog-project-build
[ 20%] Performing build step for 'cpplog-project'
[ 50%] Built target cpplog-static
[100%] Built target cpplog
[ 22%] Built target cpplog-project-build
[ 98%] Built target VC4CC
[ 98%] Linking CXX executable vc4c
~/dev/VC4C/build/src/vc4c: error while loading shared libraries: ~/dev/VC4C/build/src/libVC4CC.so.1.2: unexpected reloc type 0x03
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:108: src/vc4c] Error 127
make[2]: *** Deleting file 'src/vc4c'
make[1]: *** [CMakeFiles/Makefile2:1963: src/CMakeFiles/VC4C.dir/all] Error 2
make: *** [Makefile:172: all] Error 2

The above error occurs when taking the source from this respository on github.
In addition, I can only find get the following more detailed "error" when building the package from the AUR:

~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp: In function 'int vc4c::runProcess(const string&, std::istream*, std::ostream*, std::ostream*)':
~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp:250:18: error: ignoring return value of 'ssize_t write(int, const void*, size_t)', declared with attribute warn_unused_result [-Werror=unused-result]
  250 |             write(pipes[STD_IN][WRITE], buffer.data(), static_cast<std::size_t>(in.gcount()));
      |             ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: some warnings being treated as errors
distcc[13536] ERROR: compile (null) on localhost failed
distcc[13500] ERROR: compile ~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp on localhost failed
make[2]: *** [src/CMakeFiles/VC4CC.dir/build.make:226: src/CMakeFiles/VC4CC.dir/ProcessUtil.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....

doe300 · 2020-06-20T17:50:51Z

So the second part should be an easy fix, I will look at it tomorrow. As a work-around, putting a (void) before the write should silence the error.

The first error seems to be some kind of static/shared linking mismatch, but I don't know any details.

bronze51 · 2020-06-20T18:23:46Z

No cigar. I thought I put the (void) in the right spot?

~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp: In function 'int vc4c::runProcess(const string&, std::istream*, std::ostream*, std::ostream*)':
~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp:250:24: error: ignoring return value of 'ssize_t write(int, const void*, size_t)', declared with attribute warn_unused_result [-Werror=unused-result]
  250 |             (void)write(pipes[STD_IN][WRITE], buffer.data(), static_cast<std::size_t>(in.gcount()));
      |                   ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: some warnings being treated as errors

doe300 · 2020-06-20T18:29:16Z

Interesting. Yes, the (void) is in the right place, but somehow the compiler does not like it. You could also try with

auto dummy = write(pipes[STD_IN][WRITE], buffer.data(), static_cast<std::size_t>(in.gcount()));
(void)dummy;

bronze51 · 2020-06-20T18:43:10Z

That worked!
I did get an error but it was ignored:

[E] Sat Jun 20 13:40:00 2020: Errors in precompilation:
[E] Sat Jun 20 13:40:00 2020: distcc[21709] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[21708] ERROR: compile (null) on localhost failed with exit code 111
distcc[21707] ERROR: compile (null) on localhost failed with exit code 111

terminate called after throwing an instance of 'vc4c::CompilationError'
  what():  Pre-compilation: Error in precompilation: distcc[21709] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[21708] ERROR: compile (null) on localhost failed with exit code 111
distcc[21707] ERROR: compile (null) on localhost failed with exit code 111

[E] Sat Jun 20 13:40:00 2020: Received signal: SIGABRT

Fixes compilation error in #145

k8ieone · 2020-07-01T09:36:32Z

I separated VC4C, VC4CL and VC4CLStdLib into separate packages but I have bumped into a small problem. While building VC4C it tries to write into the directory containing VC4CLStdLib. Can I somehow tell it not to do that? Or just direct the writing part somewhere else?

[ 80%] Linking CXX executable vc4c
[E] Tue Jun 30 01:46:54 2020: Errors in precompilation:
[E] Tue Jun 30 01:46:54 2020: error: unable to open output file '/usr/local/include/vc4cl-stdlib//VC4CLStdLib.h.pch': 'Permission denied'
1 error generated.

terminate called after throwing an instance of 'vc4c::CompilationError'
  what():  Pre-compilation: Error in precompilation: error: unable to open output file '/usr/local/include/vc4cl-stdlib//VC4CLStdLib.h.pch': 'Permission denied'
1 error generated.

[E] Tue Jun 30 01:46:54 2020: Received signal: SIGABRT
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:105: src/vc4c] Error 6
make[2]: *** Deleting file 'src/vc4c'
make[2]: Leaving directory '/home/satcom886/.cache/yay/vc4c-git/src/VC4C/build'
make[1]: *** [CMakeFiles/Makefile2:2037: src/CMakeFiles/VC4C.dir/all] Error 2
make[1]: Leaving directory '/home/satcom886/.cache/yay/vc4c-git/src/VC4C/build'
make: *** [Makefile:180: all] Error 2

This is e.g. useful for packaging builds, where the pre-compilation should happen at install time. See #145

bronze51 · 2020-07-11T14:13:58Z

With khronos-ocl-icd (per doe300/VC4CL#93 (comment)) I think I made progress. I ran:

cmake -DBUILD_TESTING=OFF -DSPIRV_FRONTEND=OFF -DCLANG_FOUND=/usr/bin/clang -DVC4CL_STDLIB_PRECOMPILE=OFF ..

Perhaps VC4CL_STDLIB_PRECOMPILE is not needed in this build scenario as I got "Manually-specified variables were not used by the project: VC4CL_STDLIB_PRECOMPILE"

Next I ran make:

...
[ 98%] Built target GetGitCommit
Scanning dependencies of target VC4C
[ 98%] Building CXX object src/CMakeFiles/VC4C.dir/main.cpp.o
[ 98%] Linking CXX executable vc4c
~/dev/VC4C/build/src/vc4c: error while loading shared libraries: ~/dev/VC4C/build/src/libVC4CC.so.1.2: unexpected reloc type 0x03
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:108: src/vc4c] Error 127
make[2]: *** Deleting file 'src/vc4c'
make[1]: *** [CMakeFiles/Makefile2:1965: src/CMakeFiles/VC4C.dir/all] Error 2
make: *** [Makefile:172: all] Error 2

bronze51 · 2020-07-15T14:07:41Z

I succeeded building VC4C using the AUR build procedures! (vc4c-git)

However, I had to turn off distcc to get around:

[ 82%] Built target VC4CC
Scanning dependencies of target VC4C
[ 84%] Built target qpu_emulator
[ 98%] Built target TestVC4C
[100%] Building CXX object src/CMakeFiles/VC4C.dir/main.cpp.o
distcc[22756] (main) CRITICAL! distcc seems to have invoked itself recursively!distcc[22756] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[22755] ERROR: compile (null) on localhost failed with exit code 111
distcc[22754] (dcc_select_for_write) ERROR: IO timeout
distcc[22754] ERROR: timeout while connecting to 192.168.1.190:3632
distcc[22754] Warning: failed to distribute ~/dev/vc4c-git/src/VC4C/src/main.cpp to 192.168.1.190, running locally instead
distcc[22758] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[22757] ERROR: compile (null) on localhost failed with exit code 111
distcc[22754] ERROR: compile ~/dev/vc4c-git/src/VC4C/src/main.cpp on localhost failed with exit code 111
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:83: src/CMakeFiles/VC4C.dir/main.cpp.o] Error 111
make[1]: *** [CMakeFiles/Makefile2:2042: src/CMakeFiles/VC4C.dir/all] Error 2
make: *** [Makefile:183: all] Error 2
==> ERROR: A failure occurred in package().
Aborting...

cmake (modified by me and copied from the Arch build files) is invoked with CLANG_FOUND, but it seems to be ignored at this point of the process?

cmake "$srcdir/VC4C" -DCMAKE_BUILD_TYPE=Release -DMULTI_THREADED=true -DCLANG_FOUND=/usr/bin/clang -DVC4CL_STDLIB_DIR=/usr/local/include/vc4cl-stdlib -DVC4CL_STDLIB_PRECOMPILE=OFF -Wno-dev -DBUILD_TESTING=OFF -DSPIRV_FRONTEND=OFF

bronze51 changed the title ~~Compilation Failure on Arch Linux & distcc (pthread)~~ Compilation Failure on Arch Linux & distcc Jun 20, 2020

bronze51 mentioned this issue Jun 20, 2020

Compilation fails on RPi2B + Arch + distcc doe300/VC4CL#92

Open

doe300 added a commit that referenced this issue Jun 28, 2020

Small improvements

c401a77

Fixes compilation error in #145

doe300 added a commit that referenced this issue Jul 4, 2020

Allows to skip pre-compiling VC4CL stdlib at compile time

1bb1b55

This is e.g. useful for packaging builds, where the pre-compilation should happen at install time. See #145

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compilation Failure on Arch Linux & distcc #145

Compilation Failure on Arch Linux & distcc #145

bronze51 commented Jun 7, 2020

doe300 commented Jun 7, 2020

bronze51 commented Jun 7, 2020

doe300 commented Jun 8, 2020

k8ieone commented Jun 8, 2020

bronze51 commented Jun 10, 2020 •

edited

Loading

bronze51 commented Jun 10, 2020

doe300 commented Jun 10, 2020

bronze51 commented Jun 20, 2020 •

edited

Loading

doe300 commented Jun 20, 2020

bronze51 commented Jun 20, 2020 •

edited

Loading

doe300 commented Jun 20, 2020

bronze51 commented Jun 20, 2020

k8ieone commented Jul 1, 2020 •

edited

Loading

bronze51 commented Jul 11, 2020

bronze51 commented Jul 15, 2020

Compilation Failure on Arch Linux & distcc #145

Compilation Failure on Arch Linux & distcc #145

Comments

bronze51 commented Jun 7, 2020

doe300 commented Jun 7, 2020

bronze51 commented Jun 7, 2020

doe300 commented Jun 8, 2020

k8ieone commented Jun 8, 2020

bronze51 commented Jun 10, 2020 • edited Loading

bronze51 commented Jun 10, 2020

doe300 commented Jun 10, 2020

bronze51 commented Jun 20, 2020 • edited Loading

doe300 commented Jun 20, 2020

bronze51 commented Jun 20, 2020 • edited Loading

doe300 commented Jun 20, 2020

bronze51 commented Jun 20, 2020

k8ieone commented Jul 1, 2020 • edited Loading

bronze51 commented Jul 11, 2020

bronze51 commented Jul 15, 2020

bronze51 commented Jun 10, 2020 •

edited

Loading

bronze51 commented Jun 20, 2020 •

edited

Loading

bronze51 commented Jun 20, 2020 •

edited

Loading

k8ieone commented Jul 1, 2020 •

edited

Loading