-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kernel.stext integrity check failed due to an error in handling the p_arch_jump_label_transform_entry address on ARM64 #269
Comments
We had (a different fork of) recent LKRG run on Ubuntu 22.04.1 with kernel |
What parsing error are you referring to? If something is incorrectly read, likely you see different memory layout than LKRG which may result in such type of the problems. Additionally, LKRG synchronize with JUMP_LABEL using various locks which means it is impossible for integrity routine to not see the result of JUMP_LABEL work. It looks like you might hit some issue which is not root-cause and the patch is masking the real problem. Did you try to run under very verbose level to see what JUMP_LABEL really does? |
Sure, which I assume is @root-hardenedvault's understanding too, which is why he calls this a "workaround" and doesn't send us a PR with these changes right away. Ideally, we'd figure out the real problem and arrive at a proper fix. |
It appears that the issue is caused by a race condition. LKRG does not require any lock to be held when accessing p_db.p_jump_label.state. The panic consistently occurs during the process of updating the core text hash in arch_jump_label_transform_ret. We have also observed that p_db.p_jump_label.state is set to 1 (P_JUMP_LABEL_CORE_TEXT) when the integrity_timer calculates and compares the core text hash. It's likely that LKRG may update the core text hash while checking if it has been changed, which could lead to the race condition. Is there a mechanism in LKRG to avoid this situation? However, this cannot explain why the above patch works, since those updates would not be executed. Another scenario can trigger the panic (the similar kernel logs) is when the nftables work as a systemd service at boot time. |
Function If LKRG has this lock acquired, JUMP_LABEL engine won't modify .text section. I don't think it's a correct root-cause. |
I'm also seeing this issue, also on a Raspberry Pi 4. It occurs consistently, a few seconds after my system makes it to the login prompt. |
The reported problem with integrity verification on ARM64 (lkrg-org#269) is a result of a very tight race condition with tracepoints. Changes which simplify synchronization with JUMP_LABEL engine: f98da1b affected differently ARM64 platform which made such race possible. However, potentially the same race problem may exist on x86 and this commit fixes it and should address lkrg-org#269
The reported problem with integrity verification on ARM64 (#269) is a result of a very tight race condition with tracepoints. Changes which simplify synchronization with JUMP_LABEL engine: f98da1b affected differently ARM64 platform which made such race possible. However, potentially the same race problem may exist on x86 and this commit fixes it and should address #269
@root-hardenedvault @accelbread We think we've just fixed this issue with #294 here - can you please test and let us know? Thank you! |
I'll give it a test over the weekend, thanks! |
Unfortunately, this does not fix the issue for me :( |
@accelbread can you provide some details about the problem? What is the kernel version, How easy is to repro it? Can you recompile the LKRG with btw. I heavily tested |
I am on It is easy to reproduce. If I have default settings, a few seconds after boot, the device restarts. If I boot with "lkrg.kint_validate=1", the device does not restart a few seconds after boot, and runs fine. I can recompile and retest later with debug and logs, and get back. Seems 6.5.9 kernel is available too now so will upgrade first. I could also produce a minimal reproducing sd-card image if you'd like. |
Reproduction steps:
Hardware: Rasperry Pi 4
|
Reproduction steps:
Hardware: Rasperry Pi 4
OS: Ubuntu 22.04
We noticed that in p_arch_jump_label_transform_entry, the segment containing the destination address is parsed and the corresponding segment is updated in p_arch_jump_label_transform_ret based on the parsing result. However, parsing errors may have caused the hash of kernel.stext that needs to be updated but missed in p_arch_jump_label_transform_ret. The workaround is that adding an update operations in two places and it seem worked!
The text was updated successfully, but these errors were encountered: