-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[skip-ci] machine images #21335
Closed
Closed
[skip-ci] machine images #21335
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| workflow_dispatch: false | ||
|
|
||
| name: MACHINE IMAGE TRIAL | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| - main | ||
|
|
||
| env: | ||
| MANIFEST_ADD_ARGS: --artifact --artifact-type="" --artifact-config-type="application/vnd.oci.image.config.v1+json" --artifact-layer-type=application/vnd.oci.image.layer.v1.tar --os=linux --arch=x86_64 | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: bash -l {0} | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Install qemu dependency | ||
| run: | | ||
| sudo apt update | ||
| sudo apt -y install qemu-user-static | ||
|
|
||
| - name: Delete hostedtoolcache | ||
| run: rm -rf /opt/hostedtoolcache | ||
|
|
||
| - name: Check out code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Pull coreos-assembler image | ||
| run: podman pull quay.io/coreos-assembler/coreos-assembler | ||
|
|
||
| - name: create fcos work dir | ||
| run: mkdir ./fcos | ||
|
|
||
| - name: Cosa init | ||
| run: | | ||
| source ./contrib/podman-next/fcos-podmanimage/cosa-snippet | ||
| cd fcos | ||
| cosa init --branch podman-next https://github.com/lsm5/fedora-coreos-config | ||
|
|
||
| - name: Cosa Fetch | ||
| run: | | ||
| source ./contrib/podman-next/fcos-podmanimage/cosa-snippet | ||
| cd fcos | ||
| cosa fetch --update-lockfile | ||
|
|
||
| - name: Cosa Build Qcow2 | ||
| run: | | ||
| source ./contrib/podman-next/fcos-podmanimage/cosa-snippet | ||
| cd fcos | ||
| cosa build | ||
|
|
||
| - name: Cosa Build Hyperv | ||
| run: | | ||
| source ./contrib/podman-next/fcos-podmanimage/cosa-snippet | ||
| cd fcos | ||
| cosa buildextend-hyperv | ||
|
|
||
| - name: Cosa Build Applehv | ||
| run: | | ||
| source ./contrib/podman-next/fcos-podmanimage/cosa-snippet | ||
| cd fcos | ||
| cosa buildextend-applehv | ||
|
|
||
| - name: Cosa Compress Qcow2 | ||
| run: | | ||
| source ./contrib/podman-next/fcos-podmanimage/cosa-snippet | ||
| cd fcos | ||
| cosa compress | ||
|
|
||
| - name: Create manifest | ||
| run: buildah manifest create quay.io/podman/machine-images:5.0 | ||
|
|
||
| - name: Add Qcow2 to manifest | ||
| run: | | ||
| cd ./fcos/builds/latest/x86_64 | ||
| echo "QCOW_IMAGE_X86_64=$(ls | grep qcow2.xz)" >> $GITHUB_ENV | ||
| buildah manifest add --artifact --artifact-type="" --artifact-config-type="application/vnd.oci.image.config.v1+json" --artifact-layer-type=application/vnd.oci.image.layer.v1.tar --os=linux --arch=x86_64 --annotation "disktype=qemu" quay.io/podman/machine-images:5.0 ${{ env.QCOW_IMAGE }} | ||
|
|
||
| - name: Add Applehv to manifest | ||
| run: | | ||
| cd ./fcos/builds/latest/x86_64 | ||
| echo "APPLEHV_IMAGE=$(ls | grep applehv.x86_64.raw.gz)" >> $GITHUB_ENV | ||
| buildah manifest add --artifact --artifact-type="" --artifact-config-type="application/vnd.oci.image.config.v1+json" --artifact-layer-type=application/vnd.oci.image.layer.v1.tar --os=linux --arch=x86_64 --annotation "disktype=applehv" quay.io/podman/machine-images:5.0 ${{ env.APPLEHV_IMAGE }} | ||
|
|
||
| - name: Add Hyperv to manifest | ||
| run: | | ||
| cd ./fcos/builds/latest/x86_64 | ||
| echo "HYPERV_IMAGE=$(ls | grep hyperv.x86_64.vhdx.zip)" >> $GITHUB_ENV | ||
| buildah manifest add --artifact --artifact-type="" --artifact-config-type="application/vnd.oci.image.config.v1+json" --artifact-layer-type=application/vnd.oci.image.layer.v1.tar --os=linux --arch=x86_64 --annotation "disktype=hyperv" quay.io/podman/machine-images:5.0 ${{ env.HYPERV_IMAGE }} | ||
|
|
||
| - name: Push to Quay | ||
| uses: redhat-actions/push-to-registry@v2 | ||
| with: | ||
| image: machine-images | ||
| tags: 5.0 | ||
| registry: quay.io/podman | ||
| username: ${{ secrets.QUAY_PODMAN_USERNAME }} | ||
| password: ${{ secrets.QUAY_PODMAN_PASSWORD }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| name: MACHINE IMAGE FROM FCOS IAMGE | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| main | ||
|
|
||
| env: | ||
| IMAGE_NAME: fcos | ||
| MACHINE_IMAGE_NAME: machine-images | ||
| # IMAGE_ARCHS has to be comma separated | ||
| IMAGE_ARCHS: amd64, arm64 | ||
| IMAGE_REGISTRY: quay.io/podman | ||
| COPR_OWNER: rhcontainerbot | ||
| COPR_PROJECT: podman-next | ||
|
|
||
| jobs: | ||
| fcos-podman-next-image-build: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Install qemu dependency | ||
| run: | | ||
| sudo apt update | ||
| sudo apt -y install qemu-user-static | ||
|
|
||
| - name: Set up wait-for-copr | ||
| # Do not run on scheduled nightly builds | ||
| if: ${{ github.event_name != 'schedule' }} | ||
| run: | | ||
| pip3 install git+https://github.com/packit/wait-for-copr.git@main | ||
|
|
||
| - name: Check out code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Get short SHA from HEAD | ||
| #run: echo "SHORT_SHA=$(git rev-parse --short HEAD)" >> "$GITHUB_ENV" | ||
| # USE A KNOWN BUILT COMMIT FOR TESTING | ||
| run: echo "SHORT_SHA=f756e5db6" >> "$GITHUB_ENV" | ||
| id: short_sha | ||
|
|
||
| - name: Wait for successful podman-next build with the latest commit | ||
| # Do not run on scheduled nightly builds | ||
| if: ${{ github.event_name != 'schedule' }} | ||
| run: | | ||
| # TODO: add this in the Containerfile itself or as a --build-arg | ||
| wait-for-copr --owner ${{ env.COPR_OWNER }} --project ${{ env.COPR_PROJECT }} podman ${{ env.SHORT_SHA }} | ||
| echo "podman-next build with ${{ env.SHORT_SHA }} successful." | ||
|
|
||
| - name: Build FCOS Image | ||
| id: build_image_multiarch | ||
| # Ref: https://github.com/redhat-actions/buildah-build | ||
| uses: redhat-actions/buildah-build@v2 | ||
| with: | ||
| image: ${{ env.IMAGE_NAME }} | ||
| tags: ${{ env.COPR_PROJECT }} podman-${{ env.SHORT_SHA }} | ||
| archs: ${{ env.IMAGE_ARCHS }} | ||
| containerfiles: ./contrib/podman-next/fcos-podmanimage/Containerfile | ||
| labels: | | ||
| org.opencontainers.image.title=fcos-podman-next image | ||
| org.opencontainers.image.source=https://raw.githubusercontent.com/${{ github.repository }}/${{ github.sha }}/contrib/podman-next/fcos-podmanimage/Containerfile | ||
| org.opencontainers.image.url=https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }} | ||
| org.opencontainers.image.description=FCOS image with rpms from rhcontainerbot/podman-next copr | ||
| org.opencontainers.image.revision=${{ github.sha }} | ||
|
|
||
| #- name: Echo Outputs | ||
| #run: | | ||
| #echo "Image: ${{ steps.build_image_multiarch.outputs.image }}" | ||
| #echo "Tags: ${{ steps.build_image_multiarch.outputs.tags }}" | ||
| #echo "Tagged Image: ${{ steps.build_image_multiarch.outputs.image-with-tag }}" | ||
|
|
||
| #- name: Check images created | ||
| #run: buildah images | grep '${{ env.IMAGE_NAME }}' | ||
|
|
||
| #- name: Check image metadata | ||
| #run: | | ||
| #set -x | ||
| # COPR_PROJECT envvar is used for the `podman-next` floating tag | ||
| #buildah inspect ${{ steps.build_image_multiarch.outputs.image }}:${{ env.COPR_PROJECT }} | jq ".OCIv1.architecture" | ||
| #buildah inspect ${{ steps.build_image_multiarch.outputs.image }}:${{ env.COPR_PROJECT }} | jq ".Docker.architecture" | ||
| #buildah inspect ${{ steps.build_image_multiarch.outputs.image }}:podman-${{ env.SHORT_SHA }} | jq ".OCIv1.architecture" | ||
| #buildah inspect ${{ steps.build_image_multiarch.outputs.image }}:podman-${{ env.SHORT_SHA }} | jq ".Docker.architecture" | ||
|
|
||
| #- name: Run image | ||
| #run: podman run --privileged --rm ${{ steps.build_image_multiarch.outputs.image-with-tag }} podman system info | ||
|
|
||
|
|
||
| #- name: Push to Quay | ||
| #id: push-to-quay | ||
| # Ref: https://github.com/redhat-actions/push-to-registry | ||
| #uses: redhat-actions/push-to-registry@v2 | ||
| #with: | ||
| # image: ${{ env.MACHINE_IMAGE_NAME }} | ||
| # tags: ${{ steps.build_image_multiarch.outputs.tags }} | ||
| # registry: ${{ env.IMAGE_REGISTRY }} | ||
| # username: ${{ secrets.QUAY_PODMAN_USERNAME }} | ||
| # password: ${{ secrets.QUAY_PODMAN_PASSWORD }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| name: TRY TMT GHA | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| main | ||
|
|
||
| jobs: | ||
| fcos-podman-next-image-build: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Schedule test on Testing Farm | ||
| uses: sclorg/[email protected] | ||
| with: | ||
| api_key: ${{ secrets.TF_API_KEY }} | ||
| git_url: https://github.com/sclorg/sclorg-testing-farm | ||
| pull_request_status_name: "TEST" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shouldn't need cosa for this.
So here's how I think this would go roughly:
podman buildof an image that doesFROM: quay.io/fedora/fedora-coreos:stable, and e.g. removes moby, adds the podman RPMs, adds subscription manager, etc...osbuildto convert the container image into disk images using the same manifests we use (these are currently hosted e.g. here and here, but I think the goal is to move them out). This requires running on a system with full root privileges. For aarch64, GHA was planning to add runners but it's not clear if that has happened yet. Alternatively, you can have a self-hosted runner on e.g. AWS. Nested virt shouldn't be required AIUI since osbuild uses loopbacks instead.For the triggering, probably the simplest is polling and comparing the latest versions of the RPMs and base FCOS image against the latest pushed podman machine images (e.g. you can store the versioning in labels).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/cc @dustymabe @baude
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wrote this today for @baude: https://github.com/dustymabe/build-podman-machine-os-disks/blob/56ecb7dc0437c1abcbaf6d6a7a806d96edcdab6f/build-podman-machine-os-disks.sh#L4-L25
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so basically step 3 turns into "run this script" in an environment that can run OSBuild (i.e. run as root, selinux permissive, some osbuild RPMs installed).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
These two steps are pretty much happening already using https://github.com/containers/podman/blob/main/contrib/podman-next/fcos-podmanimage/Containerfile . This was taken from https://github.com/coreos/layering-examples/tree/main/podman-next .
subscription-manager addition isn't being done yet, I'll defer to @baude on that.
The container images built are pushed to https://quay.io/repository/podman/fcos?tab=tags .
Ack, thanks. I'll look into these two.
Ack. The trigger for the existing action right now is a commit to the main branch of podman followed by a successful rpm build at the podman-next COPR. This is done using https://github.com/packit/wait-for-copr.git . So far, we haven't been checking for image updates at quay.io/fedora/fedora-coreos:stable though IIUC that's part of upcoming work.