Description

To support multi-platform clients, it's common in the docker ecosystem to provide multi-arch images. A "multi-arch image" refers to a manifest of type application/vnd.docker.distribution.manifest.list.v2+json. These manifests refer to multiple related images which, by convention, differ only in the platform they support. I don't currently see a way to generate a multi-arch image in a vela pipeline. The easiest way I see to add this functionality is by adding functionality to the vela-docker plugin.

Example

As an example of the workflow available in the docker cli which we'd like to enable in vela, we can look to the eclipse-temurin:17 multi-arch image as an illustrative example.

$ docker manifest inspect eclipse-temurin:17
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1161,
         "digest": "sha256:36ba5c0f35130915a8dd4189cb46070904517df8a9e515bdd64bc2808e7a977b",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1161,
         "digest": "sha256:133e09bcadf3a967fb394436f553d6a06e033f8151ef0a311d41f4e722188056",
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v7"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1161,
         "digest": "sha256:76752e8157da312614847fc9c341165fb221e75c5703cbaa77b76c6d3ff04037",
         "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
         }
      },
      [...snip...]
   ]
}

We see three(+) images all referred to via the list manifest tagged as eclipse-temurin:17. If we make a couple assumptions about names for the provided digests in a local docker context, we can demonstrate how this manifest could have been created and published.

Creating platform-specific images

I believe these first three commands can already be executed in the current vela-docker plugin.

docker build --platform linux/amd64 -t eclipse-temurin:17-linux-amd64 . # assumed to produce sha256:36ba5c0f35130915a8dd4189cb46070904517df8a9e515bdd64bc2808e7a977b
docker build --platform linux/arm/v7 -t eclipse-temurin:17-linux-arm-v7 . # assumed to produce sha256:133e09bcadf3a967fb394436f553d6a06e033f8151ef0a311d41f4e722188056
docker build --platform linux/arm64/v8 -t eclipse-temurin:17-linux-arm64-v8 . # assumed to produce sha256:76752e8157da312614847fc9c341165fb221e75c5703cbaa77b76c6d3ff04037

Creating the multi-arch image

These next commands are where we need new feature development.

docker manifest create eclipse-temurin:17    \ # name the list manifest
    --amend eclipse-temurin:17-linux-amd64   \ # append platform=linux/amd64 image to the list
    --amend eclipse-temurin:17-linux-arm-v7  \ # append platform=linux/arm/v7 image to the list
    --amend eclipse-temurin:17-linux-arm64-v8  # append platform=linux/arm64/v8 image to the list
# Creating the list manifest automatically populates os and architecture, but not the variant.
# We use `docker manifest annotate` to add a variant to a specific image within the list manifest
docker manifest annotate --variant v7 eclipse-temurin:17 eclipse-temurin:17-linux-arm-v7
docker manifest annotate --variant v8 eclipse-temurin:17 eclipse-temurin:17-linux-arm64-v8
# Push the completed manifest from the local context to the registry
docker manifest push eclipse-temurin:17

Value

As developer and deployment environments become increasingly multi-architecture (windows vs linux, amd64 vs arm64, etc), providing multi-arch images greatly improves the usability of the image ecosystem. The eclipse-temurin:17 multi-arch image allows users on amd64 and arm64/v8 to both run the same command, docker run --entrypoint /bin/sh eclipse-temurin:17 -c "uname -m" and each see platform-native results.

Without a multi-arch image, the only options would be for users to reference platform-specific tags directly, or rely upon binary translation to translate a non-native image at run time. In local testing, binary translation to run amd64 JDK images on an Apple M1 host machine with an aarch64 alpine Linux VM as the docker context has been relatively unstable. For that reason, it is highly desirable to have multi-arch images available for the shared components.

Definition of Done

The work is done when it's possible to create a vela pipeline that can produce a multi-arch image. We already have the ability to create the underlying, platform-specific images. Our goal here is just to add support for stitching them together in a list manifest.

As a proposal, we could use syntax like the following to create the multi-arch image above:

- name: create_multi_arch_image
  image: docker.target.com/vela-plugins/docker:vMultiArchManifest
  pull: always
  parameters:
    registry: docker.example.org
    repo: org/eclipse-temurin
    manifest:
      name: eclipse-temurin:17
      images:
        - name: eclipse-temurin:17-linux-amd64
        - name: eclipse-temurin:17-linux-arm-v7
          variant: v7
        - name: eclipse-temurin:17-linux-amd64-v8
          variant: v8

I think it's in our best interest to have separate steps for the creation of each platform specific image, and one step for the manifest create, manifest annotate, and manifest push. Whether this syntax would properly accomplish that is an open question.

Questions, comments, and suggestions for improvement welcome.