Switch the nightly build to producing a multiarch docker image that supports both x86_64 and arm64 #133
Conversation
…upports both x86_64 and arm64
There was a problem hiding this comment.
thanks a lot for doing this! i am definitely open to this approach! the relatively small amount of changes herein are definitely small enough that it is maintainable, and the extensions look clean.
CI is red atm, but happy to press buttons whenever. Have only suggested minor things so far from a quick scan.
| RUN mkdir $PREFIX && \ | ||
| echo "$PREFIX/lib" >> /etc/ld-musl-aarch64.path && \ | ||
| ln -s /usr/include/aarch64-linux-gnu/asm /usr/include/aarch64-linux-musl/asm && \ | ||
| ln -s /usr/include/asm-generic /usr/include/aarch64-linux-musl/asm-generic && \ | ||
| ln -s /usr/include/linux /usr/include/aarch64-linux-musl/linux |
There was a problem hiding this comment.
this seems to be the most changed bit! apart from a few aarch references this is remarkably similar!
There was a problem hiding this comment.
Yeah, it might be possible to merge the two files in the future. I opted not to try just to make things easier to review.
Similarly, it should be possible to merge the nightly and stable actions by using a reusable action and just passing different parameters.
Co-authored-by: Eirik A <[email protected]>
You're welcome! I'm happy you're open to accepting it :) |
|
Ah, I didn't realise the CI stuff was already tested elsewhere! Figured there had to be a few cycles. I'll give it a go! |
Oh, yes, this took many iterations to nail down :) |
|
I just realized that we'll need some hack to force the |
🙈 i feel your pain. This must have been quite a slog with how slow the arm build is. |
yeah, that's an easy one to fix by the end. am cleaning up a few things in the ..wondering also if this is a good time to do some other breaking changes i've been wanting to do like fixing the workspace dir and maybe ripping out pq. could do a lot of this under a new tagging scheme. |
ah, i suppose the tags are already overwritten. this worked immediately! have pushed some small test stuff and will do some sanity tests tomorrow on a mac. once that is done, feel free to cherry pick my changes back into your branch and we can probably merge. |
Please don't! I am relying on it (and may not be the only one)... |
Amazing, thank you. Please let me know once you're done with your testing and I'll cherry pick |
ok, i'll leave it, but be aware that it's not in the greatest state atm. |
Interesting, good to know. Just saw you mentioning that blackdex has a newer version so would be nice to see if that can work here as well |
| ENV PATH=/root/.cargo/bin:$PREFIX/bin:$PATH \ | ||
| RUSTUP_HOME=/root/.rustup \ | ||
| CARGO_BUILD_TARGET=aarch64-unknown-linux-musl \ | ||
| CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_RUSTFLAGS="-Clink-self-contained=yes -Clinker=rust-lld -Ctarget-feature=+crt-static" \ |
There was a problem hiding this comment.
If we were to smash these dockerfiles together, having both CARGO_TARGET_X evars here for different arches should be fine, right? this is one of the few big changes
There was a problem hiding this comment.
I think so, but we will have to verify.
| # TODO: fix so that it works | ||
| RUN curl -sSL https://www.openssl.org/source/openssl-$SSL_VER.tar.gz | tar xz && \ | ||
| cd openssl-$SSL_VER && \ | ||
| CFLAGS="-mno-outline-atomics" ./Configure no-zlib no-shared -fPIC --prefix=$PREFIX --openssldir=$PREFIX/ssl linux-aarch64 && \ |
There was a problem hiding this comment.
comparing amd64 and arm64:
- CFLAGS="-mno-outline-atomics" ./Configure no-zlib no-shared -fPIC --prefix=$PREFIX --openssldir=$PREFIX/ssl linux-aarch64 && \
+ ./Configure no-zlib no-shared -fPIC --prefix=$PREFIX --openssldir=$PREFIX/ssl linux-x86_64 && \this is the other big change, do you remember what was the deal with this?
There was a problem hiding this comment.
If I recall correctly, without CFLAGS="-mno-outline-atomics", I got a bunch of linker errors for some missing symbols.
There was a problem hiding this comment.
hehe ok. sounds like a standard wtf style google session then.
|
Sanity done in #134, feel free to cherry-pick my changes and i'll merge it in :-) |
Signed-off-by: clux <[email protected]>
|
Hi @clux , just pushed the cherry-picked changes. Thanks again for moving fast on this! |
|
Awesome. Thank you so much. This was one of those big projects that I always struggled to understand how to do it, and you've done it with very elegantly. Super helpful. Thank you again. |
My pleasure! Thanks for maintaining this! Could you please force a release of a new stable image so we can get the multiarch published to dockerhub? |
sure thing, it's building from this as we speak
work is easy when people do all the work for me :-) |
It built but the |
|
Oops, we forgot to remove muslrust/.github/workflows/stable.yml Line 178 in 1149c56 |
|
yea 😞 running a faster build now without tests just to get something out the door |
|
ok, we have a stable build 🎉 |
|
beautiful, thank you! just used it to build a project on my Apple Silicon Mac :) |
Hi!
This PR reworks how the Docker image is built in a way that allows building a multiarch image that supports both x86_64 and arm64. I based it on #104 but the changes are pretty significant due to the need to use the
docker buildxflow, QEMU, etc. I based it off the example in https://docs.docker.com/build/ci/github-actions/multi-platform/.I hope you'll be open to this approach.
If so, I'll be happy to convert the stable CI job as well.this now includes thestablebuild as well since it was needed for my use-case.Happy to answer any questions about why I did things a certain way in case it is not obvious.
Thank you :)