Configure the console by default only on particular platforms #2400
Conversation
|
Skipping CI for Draft Pull Request. |
|
/test all |
1 similar comment
|
/test all |
(EDIT: nevermind, I see you covered this) |
That only triggers Prow, not Jenkins. I think CCI was restarted, and lost state. Going to try the close/reopen. |
|
webhooks are failing to be delivered.. investigating |
|
The legacy path wasn't properly setting kargs or GRUB commands in the metal or qemu images, since those don't go through Tested the contents of
Ready for review. |
|
mostly LGTM - will try to test later today |
|
This is awesome work by the way!
You touched on this but the way I think of this, the problem really parallels the bare metal case of initial provisioning and wanting debug output, particularly for the Ignition run. As well as for the initial ISO boot. Or I guess, any time Ignition runs.
Yeah, that's quite doable. But a downside of this is that if we e.g. land that code in coreos-assembler, it won't be useful for everyone using e.g. It's easy to think of This relates to coreos/fedora-coreos-tracker#235 - basically us extracting the |
|
One alternative path we could investigate for bare metal is in some cases, making it easy to "console everywhere we can" but then turn it off once the server is out of provisioning phase. I commented on this elsewhere but in OCP we recently had a large customer escalation that was greatly exacerbated by So after this lands it'd be good to document that flow of adding the console kargs, but then removing them once Ignition is done. |
This is a workaround to get console=ttyS0,115200n8 into the
aarch64 AWS image. It does so by applying the following patch
to gf-platformid:
```diff
diff --git a/usr/lib/coreos-assembler/gf-platformid b/usr/lib/coreos-assembler/gf-platformid
index 2912b322c..36d089651 100755
--- a/usr/lib/coreos-assembler/gf-platformid
+++ b/usr/lib/coreos-assembler/gf-platformid
@@ -46,7 +46,11 @@ blscfg_path=$(coreos_gf glob-expand /boot/loader/entries/ostree-*.conf)
coreos_gf download "${blscfg_path}" "${tmpd}"/bls.conf
# Remove any platformid currently there
sed -i -e 's, ignition.platform.id=[a-zA-Z0-9]*,,g' "${tmpd}"/bls.conf
-sed -i -e 's,^\(options .*\),\1 ignition.platform.id='"${platformid}"',' "${tmpd}"/bls.conf
+if [ "${platformid}" == 'aws' ]; then
+ sed -i -e 's|^\(options .*\)|\1 ignition.platform.id='"${platformid}"' console=ttyS0,115200n8|' "${tmpd}"/bls.conf
+else
+ sed -i -e 's,^\(options .*\),\1 ignition.platform.id='"${platformid}"',' "${tmpd}"/bls.conf
+fi
coreos_gf upload "${tmpd}"/bls.conf "${blscfg_path}"
if [ "$basearch" = "s390x" ] ; then
```
Once coreos/fedora-coreos-config#1181 and
coreos/coreos-assembler#2400 land then we
won't need this any longer.
This implements a fix for coreos/fedora-coreos-tracker#920
Fixes: c9036fa ("Serialize `grub-script` literally into image.json")
Soon it will both set the platform ID and configure the console.
This is an update to the serial console hack added in xyz. This needed updating because of [1] that will update and rename gf-platformid to gf-set-platform. Eventually once platforms.yaml exists everywhere we will finally be able to drop this patch. [1] coreos/coreos-assembler#2400 The decoded version of this patch looks like: ``` diff --git a/src/gf-set-platform b/src/gf-set-platform index 3b1c5ae31..df5e0f9d7 100755 --- a/src/gf-set-platform +++ b/src/gf-set-platform @@ -59,7 +59,13 @@ blscfg_path=$(coreos_gf glob-expand /boot/loader/entries/ostree-*.conf) coreos_gf download "${blscfg_path}" "${tmpd}"/bls.conf # Remove any platformid currently there sed -i -e 's, ignition.platform.id=[a-zA-Z0-9]*,,g' "${tmpd}"/bls.conf -sed -i -e '/^options / s,$, ignition.platform.id='"${platformid}"',' "${tmpd}"/bls.conf +if [ "$(coreos_gf exists /boot/coreos/platforms.json)" != "true" -a "${platformid}" == 'aws' ]; then + # Our platform is AWS and we still need the console=ttyS0 hack for the legacy + # (no platforms.yaml) path. + sed -i -e 's|^\(options .*\)|\1 ignition.platform.id='"${platformid}"' console=ttyS0,115200n8|' "${tmpd}"/bls.conf +else + sed -i -e '/^options / s,$, ignition.platform.id='"${platformid}"',' "${tmpd}"/bls.conf +fi if [ -n "$remove_kargs" ]; then # Remove existing qemu-specific kargs sed -i -e '/^options / s@ '"${remove_kargs}"'@@' "${tmpd}"/bls.conf ```
This is an update to the serial console hack added in ddd9da9. This needed updating because of [1] that will update and rename gf-platformid to gf-set-platform. Eventually once platforms.yaml exists everywhere we will finally be able to drop this patch. [1] coreos/coreos-assembler#2400 The decoded version of this patch looks like: ```diff diff --git a/src/gf-set-platform b/src/gf-set-platform index 3b1c5ae31..df5e0f9d7 100755 --- a/src/gf-set-platform +++ b/src/gf-set-platform @@ -59,7 +59,13 @@ blscfg_path=$(coreos_gf glob-expand /boot/loader/entries/ostree-*.conf) coreos_gf download "${blscfg_path}" "${tmpd}"/bls.conf # Remove any platformid currently there sed -i -e 's, ignition.platform.id=[a-zA-Z0-9]*,,g' "${tmpd}"/bls.conf -sed -i -e '/^options / s,$, ignition.platform.id='"${platformid}"',' "${tmpd}"/bls.conf +if [ "$(coreos_gf exists /boot/coreos/platforms.json)" != "true" -a "${platformid}" == 'aws' ]; then + # Our platform is AWS and we still need the console=ttyS0 hack for the legacy + # (no platforms.yaml) path. + sed -i -e 's|^\(options .*\)|\1 ignition.platform.id='"${platformid}"' console=ttyS0,115200n8|' "${tmpd}"/bls.conf +else + sed -i -e '/^options / s,$, ignition.platform.id='"${platformid}"',' "${tmpd}"/bls.conf +fi if [ -n "$remove_kargs" ]; then # Remove existing qemu-specific kargs sed -i -e '/^options / s@ '"${remove_kargs}"'@@' "${tmpd}"/bls.conf ```
|
A lot going on here, and RHCOS stuff is in bugfix mode for OCP 4.11 freeze...I think we should cut a |
|
OK actually, no reason to block on 4.11 branching, I just pushed https://github.com/coreos/coreos-assembler/tree/rhcos-4.11 |
mantle/platforms.md also has a bunch, but that's more than I want to unwind right now.
Stop specifying console= kernel arguments by default, except on specific arch/platform pairs where we know we need them. Do the same with console configuration in grub.cfg. More info is in coreos/fedora-coreos-tracker#567. Read platforms.yaml from the config repo to determine what console settings should be applied for each arch/platform. Convert the table for the current arch to JSON and copy it to /boot/coreos/platforms.json in the image, since coreos-installer needs the same data when overriding the platform ID at install time. If platforms.yaml is missing, continue to apply the previous defaults. We do this in two places: create_disk.sh configures console settings for metal and qemu images from platforms.yaml. For other platforms, gf-set-platform reads platforms.json from the image, undoes any qemu-specific settings, and applies settings for the target platform.
|
Updated! |
|
I'm good with this. If we're close to being final I can update my patch if needed and test coreos/fedora-coreos-pipeline#533 |
|
@dustymabe, yup, I think we're good to go! |
|
Local testing testing of coreos/fedora-coreos-pipeline#533 went well. I think this is now unblocked. |
|
Hi, is this planned to be part of an RHCOS release? |
|
@newkit Yes, that's the plan. |
|
@bgilbert Do you have any timeline when that’s going to happen? Would it help if I‘d create a BZ? |
|
There's no public commitment, but I'd expect it'd land soon after the functionality lands in FCOS. Note that this PR is only part of it; the overall FCOS tracking bug is coreos/fedora-coreos-tracker#567. |
Stop specifying
console=kernel arguments by default, except on specific arch/platform pairs where we know we need them. Do the same with console configuration ingrub.cfg. More info in coreos/fedora-coreos-tracker#567.Read
platforms.yamlfrom the config repo (coreos/fedora-coreos-config#1181) to determine what console settings should be applied for each arch/platform. Convert the table for the current arch to JSON and copy it to/boot/coreos/platforms.jsonin the image, since coreos-installer needs the same data when overriding the platform ID at install time (coreos/coreos-installer#605). Ifplatforms.yamlis missing, continue applying the previous defaults.We do this in two places:
create_disk.shconfigures console settings formetalandqemuimages fromplatforms.yaml. For other platforms,gf-set-platform(previouslygf-platformid) readsplatforms.jsonfrom the image, undoes anyqemu-specific settings, and applies settings for the target platform.cc @coreos/multi-arch