We need more info about cross-compiling for another architectures, like we had in the original Musl-LFS

[takusuman]

I want to at try compiling this to i386, but i think the documentation on this part is too vague.
May we can start planning documenting some things further.

At the end of my academic year i'll start pushing some notes that i wrote while compiling for x86_64, i hope it helps. 😃

[owl4ce]

Hmm .. I still don't understand if we just directly bootstrapping the musl libc along side pure Clang/LLVM.

[takusuman]

Oh, i see.
I think it would be nice.

[dslm4515]

If i understand, build llvmtools without GCC? ... as in, create a cross-toolchain without GCC that can be used to build llvmtools?

[owl4ce]

If i understand, build llvmtools without GCC? ... as in, create a cross-toolchain without GCC that can be used to build llvmtools?

100% correct.

[dslm4515]

Actually, my first attempt of CMLFS was that method. It failed right away. Either I could not build cross-tools like in MLFS, but with LLVM and no GCC ... OR ... I could not build clang in llvmtools with the modified cross-tools (that had LLVM instead of GCC). It annoyed me as I wanted CMLFS to be GCC-free from start to finish.

[dslm4515]

I got more experience with the build system for LLVM compared to when I first started this project.

I could try again to bootstrap stage0 LLVM without cross-gcc [or mussel].

[owl4ce]

Some docs:
https://llvm.org/docs/HowToCrossCompileLLVM.html

[dslm4515]

Luckily, I still got two old i686 laptops to build and test on.

I haven't gotten to testing on i386/i686 as I was upgrading my main build machine... i hit a snag when firefox stopped loading at all. Just a silent death. Took me a while to realize that librsvg ( that i built using a downloaded rust toolchain from rust-lang.org) would segfault causing firefox to die. I had to downgrade to a non-rust version of librsvg.

On another note, I can no longer build rust. Even using a rust from a previous MLFS build, rust always fails to compile itself and other packages that use/require rust. As a work around, I just repackage packages from Alpine Linux that require rust.

Edit: Long term plan is to add more support for more arches in CMLFS... I'm saving up for a Odroid-C4 as i realized there is no open-source video acceleration in the Raspberry pi 4

[owl4ce]

Looks "current" common rust issues on top musl libc. What I've seen is that rust is very problematic with musl libc. However, unfortunately I am still loyal to using the glibc host.

[dslm4515]

YES! It took me so long to get rust to work the first time. The countless failure made me hate rust. This was back when i was developing MLFS/BMLFS on an old Core 2 Duo laptop. After 8-9 hours of compilation, build just dies with errors and rust is quite cryptic in it's error output. And it's even worse the build process is not even documented [well or at all]. All i see on the rust-lang website is just use their prebuilt toolchains to get started with rust OR check with your distro for prebuilt rust...BUT I AM THE [Distro] DEVELOPER!

At the same time, I do not want to cap any of my MLFS projects to specific musl version. I would love [not really] to create patches to make rust compile, but... documentation seems rather poor...Seems too much 'user' oriented and less 'developer' friendly. This reminds me of this quote i modified:

“All right, I've been thinking, when [rust] gives you [cryptic errors], don't [give up]! Make [rust developers] take the [cryptic errors] back! Get mad! I don't want your damn [cryptic errors]! What am I supposed to do with these?"

[dslm4515]

If rust will compile on a older musl libc version... then I may have to create a 'compat' directory, such as /opt/compat, to house an older musl libc and any binaries or libraries I install from Alpine Linux. Then, for packages like rust or mozjs, i would have to hardcode (-rpath) those packages to check for libraries in /opt/compat. Just a thought. Its annoying how no rust means no building Mozilla code from source.

[dslm4515]

I am now working on cross-compiling LLVM-17.0.6 for i686 on a amd64 host. For some reason, the build method I used for compiling LLVM-15.0.6 no longer works. All sorts of strange errors like:

• Host compiler does not support -fuse-ld=lld.
• Compiler doesn't support generation of unwind tables if exception support is disabled. Building libunwind DSO with runtime dependency on C++ ABI library is not supported.
• C syntax errors for referring to a negative array element (compiler-rt/lib/builtins/int_util.h:29)
• Broken C header support (compiler cannot cannot find built-in cstdint header)

Old Method
Stage 0 LLVM will install in cgnutools but sysroot at llvmtools

1. Bootstrap build of cgnutools with mussel
2. Use mussel-built toolchain (cgnutools) to build stage0 LLVM. *This means compiling clang,lld,libunwind,libc++abi, libc++, and compiler-rt all together.
3. Use cgnutools to build stage1 LLVM with sysroot at llvmtools with stage0 LLVM. Again, this means stage 1 LLVM is compiled with clang,lld,libunwind,libc++abi, libc++, and compiler-rt all together.
4. Build enough of llvmtools to enter a chroot with stage1 LLVM
5. Build the rest of llvmtools under chroot
6. Build final system with llvmtools

New Method
Build stage 1 libc++abi, libc++, libunwind and compiler-rt individually before building stage 1 clang & lld

1. Bootstrap build of cgnutools with mussel
2. Use mussel-built toolchain (cgnutools) to build stage0 LLVM (only clang & LLD) and install in cgnutools. Do not enable support for libc++ & runtime for stage 0 clang+LLD.
3. Use stage0 clang+LLD to build stage 1 libc++abi, libc++, libunwind and compiler-rt (all three are considered part of "runtimes") individually and install in llvmtools. Also install headers to llvmtools
4. Use stage0 clang+LLD to build stage 1 clang+LLD to build against already installed stage 1 libc++abi, libc++, libunwind and compiler-rt.
5. Build enough of llvmtools to enter a chroot with stage1 LLVM
6. Build the rest of llvmtools under chroot
7. Build final system with llvmtools

Perhaps, the build system in LLVM-17.0.6 source changed? NO. I tried the old build method with LLVM 15.0.6. Same problems. When building all of LLVM together, the build wont use just built libraries and tools ... hence the separate builds of libc++abi, libc++, libunwind and compiler-rt from clang+LLD. New method builds compiler-rt "twice" for stage 1 LLVM because clang will use try to use gcc's runtime instead of compiler-rt... I think (not sure if -DCLANG_DEFAULT_RTLIB=compiler-rt is sufficient).

I assume the new method supports cross-compiling similar arches (i.e. i686 on amd64 or ... arm on arm64). I have not tried different arches like arm64 on a amd64 host. The new method has yet to be tested on native builds where host is the same as target.

[dslm4515]

When building stage2 under chroot, building all of LLVM fails just like for stage 1 LLVM

Again, stage 1 LLVM-base+clang+LLD should only be built and installed. Make sure to install llvm-lit which allows reusing the stage 1 LLVM-base to build the runtimes (libunwind, libc++abi, libc++, and compiler-rt)

Then compile & install in this order:

1. stage 2 libc++{,abi} headers. (Later it will we overridden when installing stage 2 libc++)
2. stage 2 compiler-rt
3. stage 2 libc++{,abi} and libunwind
4. rebuild stage 2 LLVM-base+clang+LLD (to fix default sysroot from /llvmtools to /)