Fetch kni-installer binary from a release payload #401

markmc · 2019-04-22T12:41:09Z

Let's consume the installer the same way stock installer builds are intended to be consumed - by extracting it from the release payload. That way we're closer to being ready to transition to using stock openshift-install in future.

In order to do this, we need to produce a container image containing the installer, reference this image from a custom release payload, and drive the install (including installer extraction) from this custom payload.

Merge extract_tools: pass security options openshift/origin#22619 or make kni/release imagestream public
Add a custom version to the custom release payload using 'oc adm release new --name'
Make it clear in the instructions what is needed when setting up a fresh namespace versus when using the existing 'kni' namespace
Handle re-running the scripts where there is a previous installer build or release payload with the same version
I've seen build_installer.sh time out waiting for the build - it has taken >15m

markmc · 2019-04-22T12:46:51Z

This PR needs openshift/origin#22619 and openshift-metal3/kni-installer#53

markmc · 2019-04-23T11:02:54Z

We now also require openshift/origin#22636

russellb · 2019-04-24T18:47:36Z

Looks like the fix we depend on is merged, via openshift/origin#22568

russellb · 2019-04-24T18:48:27Z

and the kni-installer patch is in.

One more patch needed still open: openshift/origin#22619

markmc · 2019-04-24T19:17:28Z

and the kni-installer patch is in.

One more patch needed still open: openshift/origin#22619

This one is only needed because the kni/release is private - ocp/release must not be ... need to check that out

eparis · 2019-04-24T23:24:51Z

@smarterclayton look at what these guys are pulling off! You and I have gone round and round on this.

cgwalters · 2019-04-28T21:13:37Z

Also, ultimately we need to get away from this way of caching a
pre-defined RHCOS image version - we instead need a way of caching
whatever version is driven by either openshift-install or machine
config upgrades.

i.e. openshift/installer#1399

markmc · 2019-05-07T11:02:11Z

Also, ultimately we need to get away from this way of caching a
pre-defined RHCOS image version - we instead need a way of caching
whatever version is driven by either openshift-install or machine
config upgrades.

i.e. openshift/installer#1399

Thanks, probably best discussed in openshift-metal3/kni-installer#37 - for bare-metal IPI, I don't think the user should have to do anything to download the image before kicking off the install

stbenjam · 2019-05-13T10:14:37Z

@markmc how do the build tags get set in this process? We need to build with ironic and libvirt.

markmc · 2019-05-13T14:38:07Z

@markmc how do the build tags get set in this process? We need to build with ironic and libvirt.

Yep, see https://github.com/openshift-metal3/kni-installer/blob/master/images/baremetal/Dockerfile.ci

markmc · 2019-05-13T14:39:06Z

@markmc how do the build tags get set in this process? We need to build with ironic and libvirt.

Yep, see https://github.com/openshift-metal3/kni-installer/blob/master/images/baremetal/Dockerfile.ci

And build_installer.sh has:

      dockerfilePath: images/baremetal/Dockerfile.ci

There's no straightforward way to get the installer pinned RHCOS version from an installer binary or a release payload, so let's not become reliant on this. We can easily pin dev-scripts to the correct RHCOS image each time we pin to a new release payload. Also, ultimately we need to get away from this way of caching a pre-defined RHCOS image version - we instead need a way of caching whatever version is driven by either openshift-install or machine config upgrades.

Now that we can build and publish release payloads with our installer fork, let's consume the installer the same way stock installer builds are intended to be consumed - by extracting it from the release payload. Note - we use extract --command=openshift-install to avoid roundtrip to tar.gz involved with using --tools.

Use mktemp --tmpdir so we use /tmp, remove the image-references file from tmpdir when we're done with it, move the wait_for_tag() function, and change wait_for_tag() to not wait for a specific image digest.

Allow additional images to be overwritten, rather than just the installer image. Also update the docs, and show how we are currently building with a custom baremetal-machine-controllers image.

markmc · 2019-05-17T06:36:39Z

I think this is ready to merge

There are a few improvements still listed in the checklist above, but ... well, we're relying on a custom payload now, so let's get this merged and make those improvements later if necessary

docs/release-payload.md

stbenjam

I haven't tried the process yet but I think it looks good. When rebasing the installer, is it expected to build the image and then test, or should we try to maintain some functionality in dev-scripts to build from git without a release payloud?

markmc · 2019-05-17T14:21:33Z

I haven't tried the process yet but I think it looks good. When rebasing the installer, is it expected to build the image and then test, or should we try to maintain some functionality in dev-scripts to build from git without a release payloud?

I'd be open to a dev hack to make testing the installer changes easier ... but shouldn't be the default path

stbenjam

I think it's good to merge if CI passes then, is this the only PR that's needed?

markmc · 2019-05-17T14:29:41Z

Yeah, should just work with this PR

Let's hold off for now - the deploy I kicked off earlier failed. Will look into it

derekhiggins · 2019-05-17T15:31:49Z

Build SUCCESS, see build http://10.8.144.11:8080/job/dev-tools/644/

markmc · 2019-05-17T17:33:59Z

Sorry, didn't get back to this - if CI has passed, and one person sees it working for them locally ... then I'm fine with merging

russellb · 2019-05-17T20:02:56Z

I see that CI passed, but it's not working for me locally. master does work for me. Here's what I get:

+ oc adm release extract --registry-config installer--xMN0hzCslO/pullsecret --command=openshift-install --to installer--xMN0hzCslO registry.svc.ci.openshift.org/kni/release:4.1.0-rc.3-kni.1
error: unable to read image registry.svc.ci.openshift.org/kni/release:4.1.0-rc.3-kni.1: unauthorized: authentication required

russellb · 2019-05-17T20:14:28Z

I see that CI passed, but it's not working for me locally. master does work for me. Here's what I get:

+ oc adm release extract --registry-config installer--xMN0hzCslO/pullsecret --command=openshift-install --to installer--xMN0hzCslO registry.svc.ci.openshift.org/kni/release:4.1.0-rc.3-kni.1
error: unable to read image registry.svc.ci.openshift.org/kni/release:4.1.0-rc.3-kni.1: unauthorized: authentication required

I got it to work by upgrading my client tools. Script 01 ensures a 4.1 version of oc is installed, but it seems we need a new enough one? I'm not sure how to detect that. It could just a FAQ for a bit that people need to update oc ...

stbenjam · 2019-05-17T20:42:37Z

I saw same locally as well. Could we update oc client utils on every build?

Or just base it off the build date and check

OC_DATE=$(date -d $(oc version -o json  | jq -r '.clientVersion.buildDate') +%s)
[ $OC_DATE -ge 1558047075 ] || update_oc

russellb · 2019-05-17T20:45:29Z

I saw same locally as well. Could we update oc client utils on every build?

Or just base it off the build date
OC_DATE=$(date -d $(oc version -o json  | jq -r '.clientVersion.buildDate') +%s)
[ $OC_DATE -ge 1558047075 ] || update_oc

Yeah, I like that idea. We know the current verison is new enough, at least. Want to push that up as a follow-up PR we can merge right after this one?

russellb · 2019-05-17T21:13:14Z

OK, I've got a change based on Stephen's suggestion ready that I'm just going to push straight to the repo after merging this PR since we discussed it here.

The changes in PR #401 require a newer version of oc. Make sure that one that is already is installed is at least as new as one validated to work correctly. Co-authored-by: Stephen Benjamin <[email protected]>

markmc mentioned this pull request Apr 22, 2019

Add a Dockerfile for building an installer binary image openshift-metal3/kni-installer#53

Merged

markmc force-pushed the payload branch from 1dee744 to 7c5aa12 Compare April 23, 2019 11:02

russellb self-requested a review April 25, 2019 01:35

markmc force-pushed the payload branch from 7c5aa12 to 0ac4f58 Compare May 7, 2019 10:38

russellb mentioned this pull request May 13, 2019

Changes required for 4.1.0-rc3 #523

Merged

7 tasks

markmc mentioned this pull request May 13, 2019

POC - include kubevirt's virt-operator in a custom release payload #532

Closed

markmc mentioned this pull request May 15, 2019

machine-api is failing on 4.1.0 rc.3 #537

Closed

4 tasks

markmc added 10 commits May 17, 2019 07:04

Add script for building release payloads

b1a455c

Add script for building installer images

5caa909

Include kni-install in custom release payload

c959b3c

Document the installer and release build process

abec978

Minor tweaks to prep_release.sh

df675eb

Use mktemp --tmpdir so we use /tmp, remove the image-references file from tmpdir when we're done with it, move the wait_for_tag() function, and change wait_for_tag() to not wait for a specific image digest.

Add a proper release name for our releases

bacc2fd

Make prep_release.sh a little more generic

294ef8e

Allow additional images to be overwritten, rather than just the installer image. Also update the docs, and show how we are currently building with a custom baremetal-machine-controllers image.

Pin to release payload with our installer

9b51e53

markmc force-pushed the payload branch from da30b7f to 9b51e53 Compare May 17, 2019 06:35

stbenjam reviewed May 17, 2019

View reviewed changes

docs/release-payload.md Show resolved Hide resolved

stbenjam reviewed May 17, 2019

View reviewed changes

docs/release-payload.md Show resolved Hide resolved

stbenjam reviewed May 17, 2019

View reviewed changes

stbenjam added the CI check this PR with CI label May 17, 2019

stbenjam approved these changes May 17, 2019

View reviewed changes

russellb merged commit 3a1014e into openshift-metal3:master May 17, 2019

gklein mentioned this pull request May 21, 2019

Upgrading the oc client based on ETag header changes #551

Closed

markmc deleted the payload branch August 8, 2019 12:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch kni-installer binary from a release payload #401

Fetch kni-installer binary from a release payload #401

markmc commented Apr 22, 2019 •

edited

Loading

markmc commented Apr 22, 2019

markmc commented Apr 23, 2019

russellb commented Apr 24, 2019

russellb commented Apr 24, 2019

markmc commented Apr 24, 2019

eparis commented Apr 24, 2019

cgwalters commented Apr 28, 2019

markmc commented May 7, 2019

stbenjam commented May 13, 2019

markmc commented May 13, 2019

markmc commented May 13, 2019

markmc commented May 17, 2019

stbenjam left a comment

markmc commented May 17, 2019

stbenjam left a comment

markmc commented May 17, 2019

derekhiggins commented May 17, 2019

markmc commented May 17, 2019

russellb commented May 17, 2019

russellb commented May 17, 2019

stbenjam commented May 17, 2019 •

edited

Loading

russellb commented May 17, 2019

russellb commented May 17, 2019

Fetch kni-installer binary from a release payload #401

Fetch kni-installer binary from a release payload #401

Conversation

markmc commented Apr 22, 2019 • edited Loading

markmc commented Apr 22, 2019

markmc commented Apr 23, 2019

russellb commented Apr 24, 2019

russellb commented Apr 24, 2019

markmc commented Apr 24, 2019

eparis commented Apr 24, 2019

cgwalters commented Apr 28, 2019

markmc commented May 7, 2019

stbenjam commented May 13, 2019

markmc commented May 13, 2019

markmc commented May 13, 2019

markmc commented May 17, 2019

stbenjam left a comment

Choose a reason for hiding this comment

markmc commented May 17, 2019

stbenjam left a comment

Choose a reason for hiding this comment

markmc commented May 17, 2019

derekhiggins commented May 17, 2019

markmc commented May 17, 2019

russellb commented May 17, 2019

russellb commented May 17, 2019

stbenjam commented May 17, 2019 • edited Loading

russellb commented May 17, 2019

russellb commented May 17, 2019

markmc commented Apr 22, 2019 •

edited

Loading

stbenjam commented May 17, 2019 •

edited

Loading