Port weak-modules in upstream kmod or adapt kmod to aid it

[evelikov]

Based on #253 and the 2021 presentation SLE is using weak-modules similar(?) to RHEL.

It seems that the tool calls depmod multiple times, which slows down the initrd creation process.

@mwilck have you considered having weak-modules like functionality upstream - be that in-part or in-full? Alternatively what can we add to depmod, so things are quicker? Should we add another index file so relevant look-ups don't need to re-parse all modules, or perhaps the issue is that depmod cannot dump all required information in a single run, or otherwise...

Disclaimer: I haven't looked what the tool does, how feasible it is for "porting" in kmod and if there are SLE/RHEL variations within.

[mwilck]

The idea of having this functionality upstream, and thus "standardized" somehow, is tempting. I hate the fact that the majority of out-of-tree modules are still shipped in the clumsy DKMS format. Kernel Module Packages are so much more user-friendly than that.

I think historically the notion at both Red Hat and SUSE was that upstream wasn't interested in this feature. Most people in the upstream kernel community hold that there's no such thing as a kernel ABI, and that out-of-tree modules are genuinely a bad thing that should be avoided. Only the enterprise distros, which as such are frowned upon by a lot of people upstream, really need the feature (At least that's the common notion. In truth, any distro that doesn't do rolling updates of the upstream kernel could benefit).

I realize that there's a difference between "kernel upstream" and "kmod upstream", so if you're really interested, here are a few more remarks.

First of all, "weak-modules" is just one building block of the concept. The way the modules are packaged and shipped is probably even more important. The SUSE and Red Hat approach relies heavily on rpm's ability to define arbitrary dependencies, and expressing the "kernel ABI" used by a module in terms of these dependencies. I don't know if other package managers have equivalent features. Even if they had, rewriting weak-modules in a package-manager agnositc way would be a major effort, and who would be willing to do it? SUSE and Red Hat don't need it, and others would first need to adopt the entire process. These dependencies must be generated not only when packaging the module, but also beforehand, when the kernel itself is packaged, because the kernel package contains the information about the "ABI" exposed by this kernel. IOW, to adopt the feature, a distribution must first change the way it packages its kernel, which is not a small endeavor.

The core of the weak-modules functionality is testing if adding a given module to the module tree of an existing kernel, or removing one, or replacing an existing module with a new one (by using the priority feature of the "search" configuration directive) would break any module dependencies, typically by changing symbol versions. The rest is some scripting around it, for which a shell script is not perfect but ok-ish. If the core functionality was sped up by "incremental depmod" (#253), distributions wouldn't need any additional upstream functionality.

AFAICS, upstream kmod only ever looks at the modules and symbols of a single kernel. In contrast, weak-modules looks at multiple kernels and compares their "ABI", thus it doesn't fit kmod itself too well. I guess a "weak-modules" tool could live in a separate code base in the kmod-project namespace. But would you want to start with a shell script for this purpose? I can't provide anything else. And as I said, it's really quite distro-specific and generalizing it won't be easy.

[evelikov]

Looking more specifically at the Fedora/RHEL weak-modules shows, for each kernel:

• calls depmod -n $basedir only to immediately discard ~90% of the output
• then reduces/strips the output even further adding it to an array
• in a loop, calls depmod $basedir -naeE $tmpdir/... and discards large portion of the output

I suspect that if we get the proper queries, so depmod does only what's required will also yield a benefit.

Haven't looked at the SLE variant bth and I imagine bunch of the above will also be applicable.