Build a native image for arm64 on GitHub? #44527

Stwissel · 2024-11-15T11:44:29Z

Stwissel
Nov 15, 2024

I run a build with the following GitHub action:


name: Build webapp Native
on:
  push:
    branches:
      - main
      - develop
    paths:
      - "microservices/webapp/**"
      - "libraries/**"
      - "parent/pom.xml"
  workflow_dispatch:

env:
  BD_MODULE_NAME: webapp

jobs:
  native_build:
    runs-on: ubuntu-latest
    name: Build devauth image
    steps:
      - name: GitHub container Registry login
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Checkout the code
        uses: actions/checkout@v4
        with:
          submodules: true
          token: ${{ secrets.WORKFLOW_SECRET }}

      - name: Install GraalVM
        uses: graalvm/setup-graalvm@v1
        with:
          java-version: "21"
          distribution: "mandrel"
          version: "mandrel-latest"
          github-token: ${{ secrets.GITHUB_TOKEN }}
          native-image-job-reports: "true"

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3

      - name: Build ${BD_MODULE_NAME} native executable
        env:
          BRANCH_NAME: ${{ github.head_ref || github.ref_name || 'develop' }}
        run: |
          version=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
          if [[ "$BRANCH_NAME" == "main" ]]; then
            tag="latest"
          else
            tag="$BRANCH_NAME"
          fi

          mvn -Pnative,${BD_MODULE_NAME} \
              --no-transfer-progress \
              --show-version \
              --fail-fast \
              --batch-mode \
              -DskipTests \
              -Dquarkus.native.container-build=true \
              -Dquarkus.container-image.build=true \
              -Dquarkus.container-image.push=true \
              -Dquarkus.container-image.registry=ghcr.io \
              -Dquarkus.container-image.group=beyonddemise \
              -Dquarkus.container-image.name=${BD_MODULE_NAME} \
              -Dquarkus.container-image.tag=${tag} \
              -Dquarkus.container-image.additional-tags=${version} \
              -Dquarkus.jib.platforms=linux/amd64,linux/arm64 \
              clean package

The build runs successful and I can see both amd64 and arm64 containers in the registry.

However when I try to run the container on a M1 Mac, I just get this error:

qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

What do I miss?

Answered by Stwissel

Nov 19, 2024

Apparently there is an action that would do the trick: https://github.com/marketplace/actions/run-on-architecture

View full answer

@Karm · 2024-11-15T11:44:32Z

quarkus-bot[bot]
bot Nov 15, 2024

/cc @Karm (mandrel), @galderz (mandrel), @gastaldi (m1), @geoand (jib), @zakkak (mandrel,native-image)

0 replies

Karm · 2024-11-15T11:59:47Z

Karm
Nov 15, 2024
Collaborator

Hello @Stwissel, I am not familiar with how JIB does multiarch, but it seems like the image you built is incorrectly tagged and instead of Linux aarch64 variant it's pulling Linux amd64 variant.

What happens on your aarch64 mac when you run

% podman machine start
% podman run quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 --version

What does podman inspect your_image:your_tag says?

2 replies

Stwissel Nov 15, 2024
Author

podman run quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 --version results in:

native-image 21.0.5 2024-10-15
OpenJDK Runtime Environment Mandrel-23.1.5.0-Final (build 21.0.5+11-LTS)
OpenJDK 64-Bit Server VM Mandrel-23.1.5.0-Final (build 21.0.5+11-LTS, mixed mode)

podman inspect on my image:

[
    {
        "Id": "sha256:3df571aa43dff54a7eb24d81caa039288bf464c744cf021a8253d6f9cd7d1bb7",
        "RepoTags": [
            "ghcr.io/beyonddemise/webapp:latest"
        ],
        "RepoDigests": [
            "ghcr.io/beyonddemise/webapp@sha256:60ed6e39a2ba770a5ae873d273dd26d6f55488c716218e122450b6c065f50324"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2024-11-15T04:40:44.713996401Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Image": "",
            "Volumes": {},
            "WorkingDir": "/work",
            "Entrypoint": [
                "./application"
            ],
            "OnBuild": null,
            "Labels": {
                "architecture": "aarch64",
                "build-date": "2024-10-31T12:58:20",
                "com.redhat.component": "ubi8-micro-container",
                "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI",
                "description": "Very small image which doesn't install the package manager.",
                "distribution-scope": "public",
                "io.buildah.version": "1.33.8",
                "io.k8s.description": "Very small image which doesn't install the package manager.",
                "io.k8s.display-name": "Ubi8-micro",
                "io.openshift.expose-services": "",
                "maintainer": "Red Hat, Inc.",
                "name": "ubi8/ubi-micro",
                "release": "15",
                "summary": "ubi8 micro image",
                "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/ubi-micro/images/8.10-15",
                "vcs-ref": "07e4c1d159642f87ef071ab918678c7122bf1ad2",
                "vcs-type": "git",
                "vendor": "Red Hat, Inc.",
                "version": "8.10"
            }
        },
        "Architecture": "arm64",
        "Os": "linux",
        "Size": 85322442,
        "VirtualSize": 85322442,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/f2eb2f81ae390e72fceeecd379bbca903ec13f5d2603cc0fd1cc8f6877158397/diff:/var/lib/docker/overlay2/9d1d2f131dae555fb98657099235b9267c45d3cc86bc4ed4cc7c9626ed94e584/diff:/var/lib/docker/overlay2/64d4b9b7b6d6c4acbfa57188cf40086a474320d3e1863f8d4bfdbc8d775c4af2/diff:/var/lib/docker/overlay2/a8e1a04a634a897153acf6e1b31abafc9532471c7e4a5cdd07a9db34c3d35c9e/diff",
                "MergedDir": "/var/lib/docker/overlay2/f302a613837cbc14c4a2309209112642c5af2c9fcbca2de54987605e8832f57a/merged",
                "UpperDir": "/var/lib/docker/overlay2/f302a613837cbc14c4a2309209112642c5af2c9fcbca2de54987605e8832f57a/diff",
                "WorkDir": "/var/lib/docker/overlay2/f302a613837cbc14c4a2309209112642c5af2c9fcbca2de54987605e8832f57a/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:802f2f48dfb1c4da293b57402ef340b95bc368fbc49b09fb30e6a39b8935b160",
                "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
                "sha256:8c6f1e1137b92a9b3c51cb3a5898761840e74b8a8455788e45d71d5ab851e7cd",
                "sha256:82b3dae05b66e8c7e2659b9db8927f3552df5748a40bd0975ffea42b47123723",
                "sha256:2c9e9dfbe44c374babf8cab731809bb3fff14c4cb8cd98ee729b5b2d83e20b8c"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

Stwissel Nov 15, 2024
Author

when I try to run:

docker run --rm ghcr.io/beyonddemise/webapp:latest
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

Stwissel · 2024-11-15T12:23:27Z

Stwissel
Nov 15, 2024
Author

Dockerfile for completness:

# Build the native Docker image
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.8
WORKDIR /work/
RUN chown 1001 /work \
    && chmod "g+rwX" /work \
    && chown 1001:root /work \
    && mkdir -p /local/templates \
    && chown -R 1001:root /local
COPY --chown=1001:root target/*-runner /work/application

EXPOSE 7769
USER 1001

ENTRYPOINT ["./application", "-Dquarkus.http.host=0.0.0.0"]

0 replies

Stwissel · 2024-11-15T13:21:29Z

Stwissel
Nov 15, 2024
Author

I created a reproducer based on code-with-quarkus

docker run --rm -p 8080:8080 ghcr.io/beyonddemise/code-with-quarkus:latest
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

docker inspect ghcr.io/beyonddemise/code-with-quarkus:latest

[
    {
        "Id": "sha256:4d2839e14b52fda5996f6984a4600ccd816bc5374327650646264c15d349f65b",
        "RepoTags": [
            "ghcr.io/beyonddemise/code-with-quarkus:latest"
        ],
        "RepoDigests": [
            "ghcr.io/beyonddemise/code-with-quarkus@sha256:25375cc432ebd565cf0c60b5993f7259e9477b6c52adb5116fa594d689a03966"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2024-11-15T12:58:03.789418098Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Image": "",
            "Volumes": {},
            "WorkingDir": "/work",
            "Entrypoint": [
                "./application"
            ],
            "OnBuild": null,
            "Labels": {
                "architecture": "aarch64",
                "build-date": "2024-10-31T12:58:20",
                "com.redhat.component": "ubi8-micro-container",
                "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI",
                "description": "Very small image which doesn't install the package manager.",
                "distribution-scope": "public",
                "io.buildah.version": "1.33.8",
                "io.k8s.description": "Very small image which doesn't install the package manager.",
                "io.k8s.display-name": "Ubi8-micro",
                "io.openshift.expose-services": "",
                "maintainer": "Red Hat, Inc.",
                "name": "ubi8/ubi-micro",
                "release": "15",
                "summary": "ubi8 micro image",
                "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/ubi-micro/images/8.10-15",
                "vcs-ref": "07e4c1d159642f87ef071ab918678c7122bf1ad2",
                "vcs-type": "git",
                "vendor": "Red Hat, Inc.",
                "version": "8.10"
            }
        },
        "Architecture": "arm64",
        "Os": "linux",
        "Size": 80272162,
        "VirtualSize": 80272162,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/f2eb2f81ae390e72fceeecd379bbca903ec13f5d2603cc0fd1cc8f6877158397/diff:/var/lib/docker/overlay2/9d1d2f131dae555fb98657099235b9267c45d3cc86bc4ed4cc7c9626ed94e584/diff:/var/lib/docker/overlay2/64d4b9b7b6d6c4acbfa57188cf40086a474320d3e1863f8d4bfdbc8d775c4af2/diff:/var/lib/docker/overlay2/a8e1a04a634a897153acf6e1b31abafc9532471c7e4a5cdd07a9db34c3d35c9e/diff",
                "MergedDir": "/var/lib/docker/overlay2/8b165cd198c06f517ff6978922edc44b7820f2486a3c4565783359a04eb2db5e/merged",
                "UpperDir": "/var/lib/docker/overlay2/8b165cd198c06f517ff6978922edc44b7820f2486a3c4565783359a04eb2db5e/diff",
                "WorkDir": "/var/lib/docker/overlay2/8b165cd198c06f517ff6978922edc44b7820f2486a3c4565783359a04eb2db5e/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:802f2f48dfb1c4da293b57402ef340b95bc368fbc49b09fb30e6a39b8935b160",
                "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
                "sha256:8c6f1e1137b92a9b3c51cb3a5898761840e74b8a8455788e45d71d5ab851e7cd",
                "sha256:82b3dae05b66e8c7e2659b9db8927f3552df5748a40bd0975ffea42b47123723",
                "sha256:cb3cb3fb1a40180987379b9c5c67d705820bdf4ace76957e76f768011ca5c5b8"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

3 replies

Stwissel Nov 15, 2024
Author

I then cloned the repo locally on the M1 and ran

#!/bin/bash
# run a local build
mvn -Pnative \
    --no-transfer-progress \
    --show-version \
    --fail-fast \
    --batch-mode \
    -DskipTests \
    -Dquarkus.native.container-build=true \
    -Dquarkus.container-image.build=true \
    -Dquarkus.container-image.push=false \
    -Dquarkus.container-image.registry=ghcr.io \
    -Dquarkus.container-image.group=beyonddemise \
    -Dquarkus.container-image.name=code-with-quarkus \
    -Dquarkus.container-image.tag=m1local \
    -Dquarkus.container-image.additional-tags=${version} \
    -Dquarkus.jib.platforms=linux/amd64,linux/arm64 \
    clean package

and tada: that ran nicely on the M1

Stwissel Nov 15, 2024
Author

docker inspect:

[
    {
        "Id": "sha256:88273fe2547b2c4932bda77eced90b5194a995609507e275db2390d7334021a7",
        "RepoTags": [
            "ghcr.io/beyonddemise/code-with-quarkus:m1local"
        ],
        "RepoDigests": [],
        "Parent": "",
        "Comment": "",
        "Created": "2024-11-15T13:29:57.570351Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Image": "",
            "Volumes": {},
            "WorkingDir": "/work",
            "Entrypoint": [
                "./application"
            ],
            "OnBuild": null,
            "Labels": {
                "architecture": "aarch64",
                "build-date": "2024-10-31T12:58:20",
                "com.redhat.component": "ubi8-micro-container",
                "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI",
                "description": "Very small image which doesn't install the package manager.",
                "distribution-scope": "public",
                "io.buildah.version": "1.33.8",
                "io.k8s.description": "Very small image which doesn't install the package manager.",
                "io.k8s.display-name": "Ubi8-micro",
                "io.openshift.expose-services": "",
                "maintainer": "Red Hat, Inc.",
                "name": "ubi8/ubi-micro",
                "release": "15",
                "summary": "ubi8 micro image",
                "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/ubi-micro/images/8.10-15",
                "vcs-ref": "07e4c1d159642f87ef071ab918678c7122bf1ad2",
                "vcs-type": "git",
                "vendor": "Red Hat, Inc.",
                "version": "8.10"
            }
        },
        "Architecture": "arm64",
        "Os": "linux",
        "Size": 78707562,
        "VirtualSize": 78707562,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/f2eb2f81ae390e72fceeecd379bbca903ec13f5d2603cc0fd1cc8f6877158397/diff:/var/lib/docker/overlay2/9d1d2f131dae555fb98657099235b9267c45d3cc86bc4ed4cc7c9626ed94e584/diff:/var/lib/docker/overlay2/64d4b9b7b6d6c4acbfa57188cf40086a474320d3e1863f8d4bfdbc8d775c4af2/diff:/var/lib/docker/overlay2/a8e1a04a634a897153acf6e1b31abafc9532471c7e4a5cdd07a9db34c3d35c9e/diff",
                "MergedDir": "/var/lib/docker/overlay2/f5aa6b80f2f692d203dd9e9e76b248e87a232af6fe32e7cf4b367cdba467ff0f/merged",
                "UpperDir": "/var/lib/docker/overlay2/f5aa6b80f2f692d203dd9e9e76b248e87a232af6fe32e7cf4b367cdba467ff0f/diff",
                "WorkDir": "/var/lib/docker/overlay2/f5aa6b80f2f692d203dd9e9e76b248e87a232af6fe32e7cf4b367cdba467ff0f/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:802f2f48dfb1c4da293b57402ef340b95bc368fbc49b09fb30e6a39b8935b160",
                "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
                "sha256:8c6f1e1137b92a9b3c51cb3a5898761840e74b8a8455788e45d71d5ab851e7cd",
                "sha256:82b3dae05b66e8c7e2659b9db8927f3552df5748a40bd0975ffea42b47123723",
                "sha256:9e7ad63d39a4fb547c172e6c72b3509b27738e5e4889217785754b9b51790f1c"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

Stwissel Nov 15, 2024
Author

I ran a DiffMerge, the version build on M1 is smaller (78707562 vs 80272162), the checksum differ for GraphDriver and the last layer of RootFS

zakkak · 2024-11-18T11:44:04Z

zakkak
Nov 18, 2024
Collaborator

Hi @Stwissel , as noted in the docs, quarkus.jib.platforms does not work with native images.

The feature does not work with native images, as cross-compilation is not supported.

To create cross platform binaries using native image you would need to run the build on each of the target architectures (emulation works as well).

4 replies

Stwissel Nov 18, 2024
Author

Thx for the update. On GitHub actions we have qemu, so in theory it might be possible. Would you know how to configure? Just installing obviously isn't sufficient

zakkak Nov 18, 2024
Collaborator

AFAIK we don't have anything working out of the box (or a guide for it). You will need to pass --platform=linux/arm64/v8 (or the equivalent for the platform you would like to target) to docker/podman build, which AFAIK is not possible to achieve through Quarkus properties.

Stwissel Nov 19, 2024
Author

Apparently there is an action that would do the trick: https://github.com/marketplace/actions/run-on-architecture

Answer selected by zakkak

zakkak Nov 20, 2024
Collaborator

Nice, thanks for sharing.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build a native image for arm64 on GitHub? #44527

{{title}}

Replies: 5 comments 9 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Build a native image for arm64 on GitHub? #44527

Stwissel Nov 15, 2024

Replies: 5 comments · 9 replies

quarkus-bot[bot] bot Nov 15, 2024

Karm Nov 15, 2024 Collaborator

Stwissel Nov 15, 2024 Author

Stwissel Nov 15, 2024 Author

Stwissel Nov 15, 2024 Author

Stwissel Nov 15, 2024 Author

Stwissel Nov 15, 2024 Author

Stwissel Nov 15, 2024 Author

Stwissel Nov 15, 2024 Author

zakkak Nov 18, 2024 Collaborator

Stwissel Nov 18, 2024 Author

zakkak Nov 18, 2024 Collaborator

Stwissel Nov 19, 2024 Author

zakkak Nov 20, 2024 Collaborator

Stwissel
Nov 15, 2024

Replies: 5 comments 9 replies

quarkus-bot[bot]
bot Nov 15, 2024

Karm
Nov 15, 2024
Collaborator

Stwissel Nov 15, 2024
Author

Stwissel Nov 15, 2024
Author

Stwissel
Nov 15, 2024
Author

Stwissel
Nov 15, 2024
Author

Stwissel Nov 15, 2024
Author

Stwissel Nov 15, 2024
Author

Stwissel Nov 15, 2024
Author

zakkak
Nov 18, 2024
Collaborator

Stwissel Nov 18, 2024
Author

zakkak Nov 18, 2024
Collaborator

Stwissel Nov 19, 2024
Author

zakkak Nov 20, 2024
Collaborator