Notes on how to audit a maximized flashed firmware image #107
Comments
|
hi @tlaurion
okay, since we want to create additional wiki page, All commits are signed, before being pushed into repository, 2 ways, it verifies authenticity & integrity, prior compiling:
If Heads' code base, related to specific commit, is modified, prior compiling, Commit ID will be appended to the ROM name, hashes,txt containing hashes of built files & ROM,
hashes.txt generated, after built, will only contain hashes for built files & ROM, Currently, Heads has reproducibility issues, which mean, hashes.txt will show individual hashes, of everything packed in the firmware, and also the firmware image itself. Compressed under initrd.cpio.xz:
hashes.txt under ./blobs/ section, are different from hashes.txt, generated during compiling:
Blobs (ME, GBE, IFD) will have different hashes, they are unique for each laptop:
By using blobs (ME, GBE, IFD) from Heads repository, Flashed Heads ROM, will contain MRC cache, GPG public key ring, & config.user override. Re-flashing / Updating Heads externally, provide more integrity, than internally re-flashing / updating, Internal flashing, rely on internal flashrom tool. After re-flashing, check integrity internally, by using sha256sum of internal busybox, |
|
From top post
This is coming up again and again and again: how to (reasonably) trust USB Security Dongles's flashing green light (reverse HOTP, Heads compatible dongle) / TOTP matching scanned Qr code at TOTP sealing / that TPM Disk Unlock Key passphrase isn't typed/captured in/from compromised environment Note that here, Coreboot creates an event log that can be audited through cbmem log and specifically through And again, payload being measured and sealed, by Heads, into TOTP/HOTP secret, takes care of the "meta measures" sealed from PCR 0-4 into a secret (TOTP) that can be unlocked only if the same sealed/unsealed PCR measurements are as intended (matching event log). But how to trust that the coreboot phases are not lying on recalculated reported values? We cannot reasonably trust bootblock measurement from hypothetical possible attacks. Then Heads extends those even log measurements with runtime information, including what is seen here as "user stuff" early at boot : key material, config user etc, where going to recovery shell also modifying tpm PCRs and preventing unsealing of secrets. Same for TPM DUK, which add up to what is measured and sealed. Including loaded on demand kernel modules and LUKS header. To see those in action, one would have to activate debug/tracing in recent ROM images from config settings menu and save in rom, reseal, signing and follow onscreen instructions to reseal TPM DUK if that feature is activated when setting a default boot configuration. But instructions on top post permits to validate things only manually. Could we automate another layer of verification that would require other layers to be compromised as well for an attack to succeed? Yes, we could. Heads could add an additional check. Alternative
So we could extract bootblock (cbfs being compiled from modules/flashtools not coreboot) and hash it in memory (modules/busybox sha256sum) and have those hashes as part of the detached signed digest signed and verified by gpg (modules/gnupg2). We would add an additional delay to verify bootblock prior of trusting Heads and coreboot measured boot alone. Recap. Going back in previous notes of this thread, not only cbfs would need to be tampered with in cbfs, but payload as well, to compromise also busybox, flashtools and gnupg2 modules which hashes would change (output at build time not yet verified on boot path but could also be done!) Thoughts? |
|
Should be considered under #62 |
|
What exactly is the threat model here to defend against? Somebody modifying bootblock to lie about its measurement (and also measurements of next stage)? Modifying how? From recovery shell, flashrom from within the OS, external programmer? Saving bootblock separately, to be dumped into /boot and signed makes such attack only slightly more complicated - instead of just saving "proper" hash, attacker would need to keep also the original bootblock, or at least parts that were changed. But since the attacker would modify coreboot payload anyway (after all, that's where DUK could be captured, not in the bootblock itself), faking also bootblock verification there would be quite simple (for example, modify cbfs to dump saved copy instead of the actual bootblock). Without hardware root of trust, it's hard to prevent such attack completely. One could detect it with external programmer by dumping flash content and comparing, but it isn't practical for everyday use. One thing that could raise the bar for such attack is to write-protect bootblock in SPI flash. If done properly, it would force the attacker to use external programmer, which takes significantly more time (depending on hardware, sometimes just few screws, sometimes disassembly of most of the laptop). Case opened can be detected, for example with glitter on screws. Then updates would be more complicated, but that could be solved with either/and:
The last one is IMO desirable anyway, but it's a lot of work too. And there still needs to be some other update method, for example when switching between different build origins (signed with different keys). (*) I write above "grant write access", but depending on technical details it could also mean "not protecting the area" under some conditions (as in: may require reboot into special mode, instead of reversing protection). |
This still could use some rework, I didn't do a good job reviewing it earlier. For example:
There is always only one entity that is the Root of Trust (although there may be more than one root/chain). It is implicitly (usually, unfortunately) trusted. It can be in hardware (Intel Boot Guard, AMD Hardware Validated Boot or whatever they're called today) or in software (initial part of firmware, e.g. coreboot bootblock). It is generally believed that hardware root of trust is safer, however in practice "hardware" RTM is usually just a software running on another chip, hopefully protected at least by RTV. "All stages, data and blobs" cannot be all part of root, otherwise you wouldn't have to measure anything, since by definition you trust the root.
Don't tempt me 🙂
This seems a bit backwards. Event log is only informational, it can't be trusted unless PCR values are correct (assuming that the chain of trust wasn't broken at any point since the respective root was obtained). One improvement I can think of is to somehow convey the expected PCR values (through a different channel than the binary) and compare them with actual values. They are long, checking them manually would be prone to errors, but it is still hard to get similar, but not identical hash so maybe this would work in practice. Perhaps a mobile validator app could scan the QR code and compare that to make it easier for users, but I don't know if all important PCR values would fit on it, even in binary form.
This is why you have to implicitly trust the root. There are ways to make it more trustworthy, e.g. by physically making bootblock read-only with WP pin or by using hardware RoT. In any case, you still trust that those mechanisms actually work as advertised, that they can resist attacks, that the underlying math isn't broken. If the root is broken, whole chain is. Adding more links to that chain won't improve its security, it can only make it worse.
If Heads is started by compromised earlier stages, how can you trust it to be the proper Heads, without this check tampered with? |
|
@marmarek @krystian-hebel : Decided to go with WiP/Poc code to show what is attempted to be done here at linuxboot/heads#1568 (comment) |
|
@marmarek @krystian-hebel : PoC and output, leading base to measured boot introspection and paving the way to forward sealing at linuxboot/heads#1568 (comment) and next comment |
|
Flashkeeper project accepted by NLnet : https://nlnet.nl/project/Flashkeeper/ Finishing MoU, will keep you all posted with details, and will present at QuebesOS mini-summit 2024. |
This is raw notes. This will get edited multiple times prior of having a base to create a wiki page.
The quick way, no-brainer, is to reflash the same downloaded/compiled firmware and keeping settings, as already documented at:
https://osresearch.net/Updating#reflashing-the-same-firmware
Unless flashrom itself was compromised (meaning that the payload is tampered), reflashing the same firmware image using the menu options to flash new firmware while keeping settings will reflash IFD+ME+GBE+coreboot+payload+current CBFS files. Since coreboot measures its bootblock + ramstage+romstage+payload, and then Heads measures added CBFS content, TOTP/HOTP measurements should be the same, and TOTP/HOTP unsealing of secrets should match.
But that doesn't answer the introspection need of some users to actually validate that the firmware components are actually in the expected state. And those notes, even better, if translated into code, should answer those needs.
Taking an internal backup of the firmware to be inspected externally
From Heads Recovery console, one can easily obtain actual BOARD_NAME and FW_VER that includes the bits we want to differenciate properly the actual board and flashed Heads commit ID in case we want to go back.
From Heads recovery shell, with a USB thumb drive ready to receive backup, we mount usb drive in read+write and we take a backup with proper naming scheme:
Extracting content from that firmware image for validation
Unfortunately, coreboot-utils are precompiled for a limited number of distributions today. I'll take for granted here that debian12 is installed under QubesOS (see unman's repo https://qubes.3isec.org/Templates_4.1/)
Then a quick
sudo apt install coreboot-utils binwalkwill make the tools available with/without sudo.Each Heads board's build comes with a hashes.txt file. It is either dowloadable from CircleCI artifacts, or can be seen, and copy pasted from a build's output given on screen.
Following download instructions will lead you to the hashes step of any CircleCI build: https://osresearch.net/Downloading
Now.
After having mounted our USB thumb drive under a debian-12 based qube with tools installed, from command line, having passed the USB thumb drive to our newly booted qube:
And then we work on our copied to memory image.
We can extract partitions not managed from coreboot actual versions measurements:
Here one could compare ME against what is downloaded and extracted from Heads scripts, GBE and IFD configs against what is stored under Heads tree.
But most questions are related to the firmware integrity itself, Heads itself:
Here again, a lot of stuff there.
Get build hashes for current rom
x230-hotp-maximized-Heads-v0.2.0-1296-g139ecb8_backup.rom is commit 139ecb8
We go over linuxboot/heads@139ecb8
We click grewn mark, follow the rabbit to the build for x230-hot-maximized at https://app.circleci.com/jobs/github/osresearch/heads/5448, click the "Output hashes" if this is an old build without artifacts, and copy the output of the hashes there to a local hashes.txt file we make sure to have available for our situation. We can directly save the output from CircleCI's Download Icon, which in our case leads to https://circleci.com/api/v1.1/project/github/osresearch/heads/5448/output/104/0?file=true&allocation-id=63751e76e6fb893f12bf3473-0-build%2F97AF006
From now on I consider we have that file saved into hashes.txt to be used locally.
Extract content from coreboot's payload
We now have coreboot's payload.
We extract the content with binwalk
So we have a matching initrd.cpio.xz into 2E531F.xz
We could dig more into that, and will (after all, everything hashes.txt is detailing why we have a match for the payload).
hashes.txt can be used under recovery shell to verify hashes of the initrd binaries
The paths undr hashes.txt are relative. If we have hashes.txt under USB thumb drive, from Heads recovery shell:
cd /
mount-usb
sha256sum -c /media/hashes.txt
Will give OK reports for all files that were found, including libraries and binaries and scripts user by heads, as seen under
Check bzImage
Unfortunately, this requires CircleCI bzImage since mathing can only be made against decompressed binary.
After our binwalk --extract command:
Download bzImage from CircleCI artifact
The text was updated successfully, but these errors were encountered: