Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

musl-cross-make: bump musl (1.2.0 -> 1.2.5) + rename module to reflect reality, bump newt ( whiptail:0.52.20 -> 0.52.24) to fix crash with newer musl #1841

Merged
merged 5 commits into from
Nov 8, 2024

Conversation

tlaurion
Copy link
Collaborator

@tlaurion tlaurion commented Nov 6, 2024

This bumps musl-cross-make to 1.2.5+ and containes fixes for all boards to build.

Talos2: There is a bug with skiboot made obvious with absence of patch needed for skiboot to build with newer muslc, showing that skiboot builds with muslc, not coreboot buildstack. This is a temporary and ugly fix

TODO:

@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 6, 2024

Created Dasharo/dasharo-issues#1126 for long time and more proper fix. This PR includes a patch for Talos, nothing more.

…om ~0.9.9+->~0.9.10+ (musl 1.2.0 -> 1.2.5)

Signed-off-by: Thierry Laurion <[email protected]>
…S against newer mulslc (Doesn't use coreboot buildstack, obviously)

Signed-off-by: Thierry Laurion <[email protected]>
@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 7, 2024

#1840 builds and is merged, rebasing on master

@tlaurion tlaurion marked this pull request as ready for review November 7, 2024 00:13
…l crash with newer muslc + parallel builds

popt havent't changed.

Signed-off-by: Thierry Laurion <[email protected]>
@tlaurion tlaurion marked this pull request as draft November 7, 2024 15:23
@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 7, 2024

tested (factory-reset, passphrase change, reencryption, totp.hotp, TPM DUK, boot to final os:

  • qemu-coreboot-whiptail-tpm1 : found issue with whiptail (newt): version bumped.
  • qemu-coreboot-fbwhiptail-tpm2
  • x230-hotp-maximized
  • talos-2 (new ram modules: doesn't boot anymore, sigh... Put untested for now?)

@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 7, 2024

size.txt analysis per linuxboot/heads-wiki#165 for x220-hotp-maximized (on the verge of getting umaintained...) But this is a gain, not a loss! See below.

TLDR: the payload size is actually smaller, where all other numbers show counter intuitive increase in size.
Seems like cpio (heads.cpio, tools.cpio, modules.cpio) are more compressible now then before on master!

coreboot stitching output

This PR:

"/home/user/heads/build/x86/coreboot-24.02.01/x220-hotp-maximized/cbfstool" "/home/user/heads/build/x86/coreboot-24.02.01/x220-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs_master_header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           98160 none
cpu_microcode_blob.bin         0x18080    microcode       26624 none
fallback/ramstage              0x1e8c0    stage          148872 LZMA (323424 decompressed)
config                         0x42ec0    raw              3359 LZMA (10703 decompressed)
revision                       0x43c40    raw               724 none
build_info                     0x43f40    raw               101 none
bootsplash.jpg                 0x44000    bootsplash      43282 none
fallback/dsdt.aml              0x4e940    raw             14715 none
vbt.bin                        0x52300    raw              1400 LZMA (3985 decompressed)
cmos_layout.bin                0x528c0    cmos_layout      1976 none
fallback/postcar               0x530c0    stage           29980 none
fallback/payload               0x5a640    simple elf    7705386 none
(empty)                        0x7b39c0   null            90084 none
bootblock                      0x7c99c0   bootblock       25600 none
2024-11-07 16:35:17+00:00 INSTALL   build/x86/coreboot-24.02.01/x220-hotp-maximized/coreboot.rom => build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom
44e0d60415fe3fd3ffa1869111f6702dd6952d50097badae701428624b2c7fd1  build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom
 8388608:build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom
rm -rf "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg"
mkdir -p "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg"
cp "/home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom" "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg/"
cd "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg" && sha256sum "heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom" >sha256sum.txt
cd "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg" && zip -9 "/home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.zip" "heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom" sha256sum.txt
  adding: heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom (deflated 4%)
  adding: sha256sum.txt (deflated 15%)
44e0d60415fe3fd3ffa1869111f6702dd6952d50097badae701428624b2c7fd1  /home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom
 8388608:/home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom

Master:

"/home/user/heads/build/x86/coreboot-24.02.01/x220-hotp-maximized/cbfstool" "/home/user/heads/build/x86/coreboot-24.02.01/x220-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs_master_header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           98160 none
cpu_microcode_blob.bin         0x18080    microcode       26624 none
fallback/ramstage              0x1e8c0    stage          148873 LZMA (323424 decompressed)
config                         0x42ec0    raw              3359 LZMA (10703 decompressed)
revision                       0x43c40    raw               724 none
build_info                     0x43f40    raw               101 none
bootsplash.jpg                 0x44000    bootsplash      43282 none
fallback/dsdt.aml              0x4e940    raw             14715 none
vbt.bin                        0x52300    raw              1400 LZMA (3985 decompressed)
cmos_layout.bin                0x528c0    cmos_layout      1976 none
fallback/postcar               0x530c0    stage           29980 none
fallback/payload               0x5a640    simple elf    7707082 none
(empty)                        0x7b4040   null            88420 none
bootblock                      0x7c99c0   bootblock       25600 none
2024-11-07 16:35:10+00:00 INSTALL   build/x86/coreboot-24.02.01/x220-hotp-maximized/coreboot.rom => build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom
fe19904ccfe7810b244f8d63a77eb128829926b9ab68f4ec673438fa5d9deeb7  build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom
 8388608:build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom
rm -rf "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg"
mkdir -p "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg"
cp "/home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom" "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg/"
cd "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg" && sha256sum "heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom" >sha256sum.txt
cd "/home/user/heads/build/x86/x220-hotp-maximized/update_pkg" && zip -9 "/home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.zip" "heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom" sha256sum.txt
  adding: heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom (deflated 4%)
  adding: sha256sum.txt (deflated 14%)
fe19904ccfe7810b244f8d63a77eb128829926b9ab68f4ec673438fa5d9deeb7  /home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom

switch to newer musl + newer newt (whiptail), change of empty space in CBFS:
master-pr = change
88420-90084 = gain of -1664 with this PR.


size.txt output diff

user@heads-tests-deb12-nix:~/heads$ diff -u build/x86/x220-hotp-maximized/sizes.txt ~/QubesIncoming/heads-tests-deb12-nix-new_musl/sizes.txt 
--- build/x86/x220-hotp-maximized/sizes.txt	2024-11-07 11:35:11.355117779 -0500
+++ /home/user/QubesIncoming/heads-tests-deb12-nix-new_musl/sizes.txt	2024-11-07 11:35:17.741118000 -0500
@@ -1,77 +1,77 @@
-2024-11-07 16:34:49+00:00 9d656fceb56f701a626aea5ceb5a5fab73720bb0 clean
- 2820000:/home/user/heads/build/x86/x220-hotp-maximized/bzImage
+2024-11-07 16:34:54+00:00 2a8cc11a46a31b95b243209936ea04676f16363f clean
+ 2822912:/home/user/heads/build/x86/x220-hotp-maximized/bzImage
   930816:/home/user/heads/build/x86/x220-hotp-maximized/modules.cpio
 -----
     9968:./lib/modules/cdc_eem.ko
-   69080:./lib/modules/ehci-hcd.ko
-  133944:./lib/modules/usb-storage.ko
-  180696:./lib/modules/xhci-hcd.ko
-   41264:./lib/modules/cdc_ncm.ko
-   14936:./lib/modules/mii.ko
-   22744:./lib/modules/cdc_ether.ko
-   52800:./lib/modules/usbnet.ko
+   69192:./lib/modules/ehci-hcd.ko
+  133920:./lib/modules/usb-storage.ko
+  180984:./lib/modules/xhci-hcd.ko
+   41128:./lib/modules/cdc_ncm.ko
+   14888:./lib/modules/mii.ko
+   22696:./lib/modules/cdc_ether.ko
+   52976:./lib/modules/usbnet.ko
    11736:./lib/modules/ehci-pci.ko
-   41200:./lib/modules/usbhid.ko
-  336360:./lib/modules/e1000e.ko
-   13648:./lib/modules/xhci-pci.ko
+   41264:./lib/modules/usbhid.ko
+  336264:./lib/modules/e1000e.ko
+   13600:./lib/modules/xhci-pci.ko
 -----
-13282304:/home/user/heads/build/x86/x220-hotp-maximized/tools.cpio
+13302784:/home/user/heads/build/x86/x220-hotp-maximized/tools.cpio
 -----
-  171032:./lib/libpng16.so.16
+  171040:./lib/libpng16.so.16
   300528:./lib/libdevmapper.so.1.02
- 1194136:./lib/libgcrypt.so.20
+ 1198224:./lib/libgcrypt.so.20
   563760:./lib/libpixman-1.so.0
-   10080:./lib/libaio.so.1
+   10088:./lib/libaio.so.1
    48128:./lib/libpopt.so.0
-  491024:./lib/libcairo.so.2
+  495128:./lib/libcairo.so.2
   104744:./lib/libz.so.1
-  314728:./lib/libtpm.so
-  207216:./lib/libksba.so.8
+  314736:./lib/libtpm.so
+  207224:./lib/libksba.so.8
    22840:./lib/libuuid.so.1
   316224:./lib/libblkid.so.1
-   51624:./lib/libjson-c.so.5
+   51608:./lib/libjson-c.so.5
    84592:./lib/libusb-1.0.so.0
    18664:./lib/libnpth.so.0
   662056:./lib/libc.so
   125728:./lib/libgpg-error.so.0
-  441728:./lib/libcryptsetup.so.12
-   44008:./lib/libpci.so.3.5.4
+  441640:./lib/libcryptsetup.so.12
+   43992:./lib/libpci.so.3.5.4
    39240:./lib/libqrencode.so.3
-   44008:./lib/libpci.so.3
-   64464:./lib/libassuan.so.0
-  288272:./lib/libmbedcrypto.so.0
+   43992:./lib/libpci.so.3
+   64456:./lib/libassuan.so.0
+  288280:./lib/libmbedcrypto.so.0
    10064:./bin/io386
-  744472:./bin/flashprog
-   45328:./bin/veritysetup
-  922016:./bin/bash
-  828152:./bin/gpg
+  744480:./bin/flashprog
+   45320:./bin/veritysetup
+  930200:./bin/bash
+  832240:./bin/gpg
       35:./bin/whiptail
-   10096:./bin/poke
-   14208:./bin/uefi
-   44024:./bin/pinentry-tty
+   10104:./bin/poke
+   14216:./bin/uefi
+   44016:./bin/pinentry-tty
   119136:./bin/dmsetup
  2248512:./bin/lvm
    39528:./bin/cbmem
-    5912:./bin/hotp
-  344000:./bin/gpg-agent
+    5920:./bin/hotp
+  343992:./bin/gpg-agent
    43608:./bin/fbwhiptail
-  154616:./bin/kexec
-  532360:./bin/busybox
-   10088:./bin/peek
+  154608:./bin/kexec
+  532376:./bin/busybox
+   10096:./bin/peek
    43872:./bin/fsck.exfat
-   73464:./bin/lspci
-  118424:./bin/tpm
+   73448:./bin/lspci
+  118352:./bin/tpm
    38920:./bin/mkfs.exfat
     1087:./bin/hotp_initialize
-    5904:./bin/qrenc
-   14240:./bin/cbfs
-  169472:./bin/cryptsetup
-  450808:./bin/scdaemon
-   42296:./bin/hotp_verification
-   10032:./bin/totp
-  174352:./bin/zstd-decompress
-   18336:./bin/flashtool
-  363288:./bin/mke2fs
+    5912:./bin/qrenc
+   14248:./bin/cbfs
+  169464:./bin/cryptsetup
+  450800:./bin/scdaemon
+   42320:./bin/hotp_verification
+   10040:./bin/totp
+  174360:./bin/zstd-decompress
+   18344:./bin/flashtool
+  363272:./bin/mke2fs
     1740:./etc/terminfo/l/linux
      733:./etc/config
 -----
@@ -155,5 +155,5 @@
      924:./sbin/config-dhcp.sh
     1840:./sbin/insmod
 -----
- 4897792:build/x86/x220-hotp-maximized/initrd.cpio.xz
- 8388608:/home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2410-g9d656fc.rom
+ 4893184:build/x86/x220-hotp-maximized/initrd.cpio.xz
+ 8388608:/home/user/heads/build/x86/x220-hotp-maximized/heads-x220-hotp-maximized-v0.2.0-2413-g2a8cc11.rom

@tlaurion tlaurion marked this pull request as ready for review November 7, 2024 17:12
@tlaurion tlaurion changed the title musl-cross-make: bump musl (1.2.0 -> 1.2.5) + rename module to reflect reality musl-cross-make: bump musl (1.2.0 -> 1.2.5) + rename module to reflect reality, bump newt ( whiptail:0.52.20 -> 0.52.24) to fix crash with newer musl Nov 7, 2024
@tlaurion tlaurion marked this pull request as draft November 7, 2024 17:14
@tlaurion tlaurion marked this pull request as ready for review November 7, 2024 17:15
Copy link
Collaborator

@JonathonHall-Purism JonathonHall-Purism left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Size reduction is surprising indeed but no complaints here 🙂

@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 7, 2024

Not sure what to think of Nitrokey/nitrokey-hotp-verification#37 badluck?

@JonathonHall-Purism
Copy link
Collaborator

@tlaurion It was a one-off occurrence, not reproducible when running this MR again? If so it seems most likely unrelated. Hard to be 100% sure without identifying the actual cause, but given the nature of the issue and the changes here I think it is unlikely enough that we can merge.

@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 7, 2024

@tlaurion It was a one-off occurrence, not reproducible when running this MR again? If so it seems most likely unrelated. Hard to be 100% sure without identifying the actual cause, but given the nature of the issue and the changes here I think it is unlikely enough that we can merge.

This is HOTP having been sealed in nk3 (can only be done once from Heads when nk3 secure element secret app has no PIN defined (in factory mode as shipped by Nitrokey), needs resetting through nitropy on second computer as of now) and was detailed under Nitrokey/nitrokey-hotp-verification#36 (comment)

Will open an issue under Heads, from my understanding:

An image is 1000 words they say.

signal-2024-11-07-124406

Screenshot:
signal-2024-11-07-123429

@JonathonHall-Purism
Copy link
Collaborator

Thanks, makes sense. Not related to this change, let's merge 🚢

@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 7, 2024

Putting again in draft mode, not quite sure why talos-2 doens't build, but there are hardcoded strings in code fro board name == talos-2 so will need fix for that too.

@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 8, 2024

Seems like check for talos-2 board inside of Makefile was cultprit of thought broken #1843. Seems like 18fcc73 fixes it, waiting for clean CircleCI build to succeed (since Makefile changed, no cache reused per caching rules)

@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 8, 2024

Will invert last two commits for clarity before merging this: talos-2 untested because new ram issues here, also borked my nvme disk with past fixes testing now merged #1541 (which has open bug since we need to bump kernel version which needs patching which is #1844

@tlaurion tlaurion marked this pull request as ready for review November 8, 2024 17:51
Copy link
Collaborator

@JonathonHall-Purism JonathonHall-Purism left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest changes, looks good to me. I looked over the talos-2 build directory changes pretty closely, especially the Linux config change, all looks right to me.

…name can change over time

Signed-off-by: Thierry Laurion <[email protected]>
… to test nor report issues for this unknowingly used board prior of feature freeze planned for 2024-11-20

repro: helper used
time docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:latest -- make BOARD=talos-2 board.move_tested_to_untested

Signed-off-by: Thierry Laurion <[email protected]>
@tlaurion
Copy link
Collaborator Author

tlaurion commented Nov 8, 2024

Succeeded and saved cache. Inverting last two commits.

@tlaurion tlaurion merged commit cd683b1 into linuxboot:master Nov 8, 2024
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants