What exactly is being authenticated in arc_hdr_authenticate()?

[adamdmoss]

Ask your question!

There's a code path in arc_hdr_authenticate() with this comment:

/*
 * The MAC is calculated on the compressed data that is stored on disk.
 * However, if compressed arc is disabled we will only have the
 * decompressed data available to us now. Compress it into a temporary
 * abd so we can verify the MAC. The performance overhead of this will
 * be relatively low, since most objects in an encrypted objset will
 * be encrypted (instead of authenticated) anyway.
 */

What is this authentication even used for - is this some sort of legacy feature or is it part of authentication for the encryption feature? If this case is as rare as the comment suggests then it'd be more robust and adequately performant to do the MAC on the decompressed data... or never allow records which need authentication to be compressed at all. (But that switchover would break MACs too, once. 😁 )

If I'm understanding that correctly, it seems awfully fragile to re-compress an old record with new code (or vice-versa) and require that the compressed result be bit-identical. This can unsubtly break when upgrading compression code (like the upcoming zstd upgrade and a WIP lz4 upgrade I've been playing with), and more subtly break if the 12.5% compression cutoff in zio_compress_data() is ever changed (which I'm also looking at making tweakable per dataset).

Which portion of the codebase does your question involve?

arc.c : arc_hdr_authenticate()

Additional context

Thanks.

[behlendorf]

@tcaputi would you mind commenting on this.

[gmelikov]

You may find some additional context about compression, it's threshold and encryption here in comments #9416 , I've already tried to implement 12.5% compression cutoff change, and I've got a problem with backward compatibility. (Looks like you already saw it, but I leave the link here just in case)

[ahrens]

@tcaputi can give a more detailed answer, but here's what I know: This is part of authentication for the encryption feature. I agree that depending on re-compressing generating identical bits is fragile. Thankfully this is only relevant if compressed ARC is disabled. It's enabled by default, and we're considering making it mandatory.

[adamdmoss]

Thanks very much for the replies.

How might I test this path - is it, like, just used as part of the zfs load-key verification stage? In which case is it performed against a special sort of non-file object or something else that can be used to distinguish this case?

Cheers.