lints: add check for non-utf-8 filesystem content

[allisonkarlitskaya]

As mentioned in #975, we should add a lint to ensure that all filenames are UTF-8, giving nice errors (with full pathnames) in the event that we encounter invalid filenames.

We also check symlink targets.

Add a unit test that tries various valid and invalid scenarios.

[allisonkarlitskaya]

From what I can see of how we get our root Dir reference in cli.rs, and the recommended instructions for running this in a RUN command in the Containerfile, I think this might end up walking into /dev, /sys, /proc and so on as well...

On the other hand, any bind mounts that are present might prevent us from scanning the underlying content. Imagine some scenario where /etc/resolv.conf in the image was a symlink that contained non-utf8 character, but we wouldn't be able to access that because it gets bind-mounted-over for the duration of the RUN command.

Maybe we should do a temporary bind mount of (only) the rootfs for purposes of linting?

Thoughts?

[cgwalters]

From what I can see of how we get our root Dir reference in cli.rs, and the recommended instructions for running this in a RUN command in the Containerfile, I think this might end up walking into /dev, /sys, /proc and so on as well...

Yeah we should avoid that for sure. There's relevant similar code in e.g. remove_all_on_mount_recurse

[cgwalters]

LGTM apart from the mount traversal you noted, thanks!

[allisonkarlitskaya]

I don't really trust checking dev vs. rootdev anymore: we've had some reports of that that show that overlayfs with composefs can produce strange effects in this case... like in containers/composefs-rs#41 (comment)

The overlayfs docs themselves warn about this:

While directories will report an st_dev from the overlay-filesystem, non-directory objects may report an st_dev from the lower filesystem or upper filesystem that is providing the object. Similarly st_ino will only be unique when combined with st_dev, and both of these can change over the lifetime of a non-directory object. Many applications and tools ignore these values and will not be affected.

https://docs.kernel.org/filesystems/overlayfs.html

The openat2 NO_XDEV flag approach seems kinda okay-ish I guess... does cap-std have support for this?

I think the best thing we could do here is some trickery with cloning the root mount to an fd using the new mount API and then iterating through that fd...

[allisonkarlitskaya]

Okay, "some trickery" isn't very tricky at all. It mostly looks like just using open_tree() without the AT_RECURSIVE flag. rustix binds this, so I'll take a look.

https://github.com/brauner/man-pages-md/blob/main/open_tree.md

[cgwalters]

I don't really trust checking dev vs. rootdev anymore: we've had some reports of that that show that overlayfs with composefs can produce strange effects in this case... like in containers/composefs-rs#41 (comment)

Yeah, fair.

OK so another approach which is also used in the referenced code is this: #984

The openat2 NO_XDEV flag approach seems kinda okay-ish I guess... does cap-std have support for this?

Not directly but we can also just use rustix for stuff like this too.

[cgwalters]

Okay, "some trickery" isn't very tricky at all. It mostly looks like just using open_tree() without the AT_RECURSIVE flag. rustix binds this, so I'll take a look.

That is awesome! I didn't know that was usable unprivileged. Indeed it's by far the best approach. I noticed this wasn't doc'd so ➡️ brauner/man-pages-md#15

[cgwalters]

Nice thanks!

[cgwalters]

Ah CI needs some #[cfg(feature = "install")] massaging...I am quite tempted again to drop that feature or just scope it to to-disk

[cgwalters]

After a live chat we decided to just rely on openat2(.. NO_XDEV), updated for that.

[cgwalters]

Also noting here just so it doesn't get lost:

We also discussed having container lint --root /path/to/root which would be usable in a multi-stage build like this:

FROM someimage as rootfs
...


FROM otherimage as linter
RUN --mount=from=rootfs,target=/rootfs bootc container lint --rootfs=/rootfs

An advantage of this approach is that we'd be able to scan the filesystem tree without having container runtime state (/proc, /dev) overmounted at all...which would actually allow us to e.g. also lint against device nodes.

But...the ergonomics would be obviously less good.

And ultimately the most powerful way to do this is to have a container scanner that runs outside of the running image, which would allow us to lint the actual image structure itself (e.g. tiny layers that should be coalesced). Or of course, a container -> container optimizer.