add support for erofs as root filesystem format #379
Conversation
Signed-off-by: Ben Cressey <[email protected]>
erofs is a relatively new filesystem, especially when compared with ext4. It shows promising results in terms of improved boot times, but its behavior at runtime on fully loaded nodes is more difficult to predict. Mark the image feature as experimental for now. This allows it to be enabled by downstream variants to facilitate additional experiments, without immediately endorsing it for production use. Signed-off-by: Ben Cressey <[email protected]>
Signed-off-by: Ben Cressey <[email protected]>
bcressey
force-pushed
the
erofs-image-feature
branch
from
d8d1b90 to
d1baea7
Compare
| # shellcheck disable=SC2312 # mapfile is validated elsewhere | ||
| mapfile -t new_root_artifacts <<<"$(find "${ROOT_MOUNT}" -type f)" | ||
|
|
||
| # The reason we check index 0 rather than the mapfile length is if `find` fails | ||
| # to find an artifact the heredoc to mapfile will assign empty output to 0. | ||
| if [[ -z "${new_root_artifacts[0]}" ]]; then | ||
| echo "no new root artifacts found" >&2 | ||
| exit 1 | ||
| else | ||
| # Write files from the root mount to the root image. | ||
| ROOT_DEBUGFS_STDERR="${WORKDIR}/root.err" | ||
| for artifact in "${new_root_artifacts[@]}"; do | ||
| cat <<EOF | debugfs -w -f - "${ROOT_IMAGE}" 2>>"${ROOT_DEBUGFS_STDERR}" | ||
| rm ${artifact#"${ROOT_MOUNT}"} | ||
| write ${artifact} ${artifact#"${ROOT_MOUNT}"} | ||
| ea_set ${artifact#"${ROOT_MOUNT}"} security.selinux system_u:object_r:os_t:s0 | ||
| EOF | ||
| done | ||
| check_debugfs_errors "${ROOT_DEBUGFS_STDERR}" | ||
| done | ||
| check_debugfs_errors "${ROOT_DEBUGFS_STDERR}" | ||
| fi |
There was a problem hiding this comment.
Since we're just completely remaking the root filesystem if it's erofs, should we follow the same workflow for ext4?
There was a problem hiding this comment.
For erofs, extracting the filesystem is a supported feature of fsck.erofs. For ext4, I'm more confident that debugfs will do what we want, since I'm not aware of a way to do full filesystem extraction using tools from e2fsprogs.
There was a problem hiding this comment.
The erofs paper mentions support for "image patching" in 4.3, but I didn't find a way to do that with erofs-utils.
|
One question tho, what was the final size of the root filesystem? |
For |
Issue number:
N/A
Description of changes:
Add the
erofs-root-partitionimage feature. This is still considered experimental, but using erofs for the root filesystem has shown significant improvements in overall boot time. I'd like to land this implementation, and the associated changes in the core kit, to facilitate further experiments.I tried many different settings for
mkfs.erofsand eventually settled on the ones here. The physical cluster size made the largest difference. I tested various extended options -dedupe,fragments,all-fragments, andztailpacking- but left those out to simplify the initial implementation, and to avoid features that the 6.1 kernel labels as experimental.Testing done:
Built
aws-devandaws-k8s-1.30images with the new feature flag enabled. Confirmed thatrepack-variantalso worked.Using some tools provided by @cbgbt, I compared the boot times for
aws-k8s-1.30AMIs as of the v1.22.0 release, with the current ext4 root filesystem and the experimental erofs root filesystem.Terms of contribution:
By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.