Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add depmod -M option to replace kmod.pc #127

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

hramrach
Copy link
Contributor

Move ligkmod features from kmod.pc to libkmod.pc

sysconfdir, distconfdir, module_compressions, and module_signatures are
libkmod features, these should be in libkmod.pc, not kmod.pc.

module_directory is special: While libkmod has a default for when the
directory is not specified depmod does spefcify it, and has its own
embetted value.

Technically it's possible to have kmod and libkmod with mismatched
values due to them being built separately. Also on a binary OS
distribution the libkmod.pc would be installed only with libkmod
development headers, not the tool.

depmod: Add -M parameter to print the value of MODULE_DIRECTORY

This is the only value left in kmod.pc.

It is an overkill to create a pkgconfig file for one value.

Also this file was initially introduced, then moved to a different
loaction because its purpose is misunderstood. Using it is awkward
because the binary search path and pkgconfig path must be matched in
case there is a possibility that multiple builds of kmod exist.

As to why multiple build of kmod might exist on a system

 - custom test build
 - build of kbuild that matches some target system which has different
   settings for sysconfdit, distconfdir, or module_directory
 - per-architecture tools on 'multiarch' system that is set up for
   building binaries/boot images for arbitrary architectures
   - when these use a filesystem scheme in which per-architecture
     libraries and modules are stored in per-architecture directories
     Linux sticks out like a sore thumb with all modules in /lib

While in the past it was fine to have One Kmod To Rule Them All because
everything was hardcoded to a fixed value (and all compression methods
and signature options could be built at the same time) this is no longer
the case.

Values for sysconfdir, distconfdir, and module_directory may differ
between systems, and thus a build can be hosted on a system with one set
of values but targeted for a system with different set of values.

The -m option is provided to make it possible to override
module_directory.

However, for the modules to be installed in the correct location by
kernel build the kernel build needs a way to determine the default value
of module_directory, which this -M option provides.

With a corresponding change to the Linux kbuild system installing
modules should just work for both host with arbitrary module_directory
and target for which correctly configured target-depmod is available and
used during build.

There is always the option to pass arbitrary value as argument to the -m
option to build modules for a target system that uses different
module_directory, or to move the installed modules after the fact.
However, that requires somehow examining the target system out-of-band
to determine the correct value, and hardcode such value in a build
script. When working with multiple different targets regularly using a
target-specific depmod is a way to manage the depmod build-time settings
in a standardized way that works equally for other tools.

@evelikov
Copy link
Collaborator

Is there a discussion which inspired this series? I'm not sure I follow why it's practically needed (mind you, I'm mid coffee).

AFAICT this series will regress existing users which depend on kmod.pc - not sure how many there are and to what extend we should care though. Would it make sense to duplicate the values in libkmod.pc instead?

depmod does not have any "get" options and as-is this one can be easily confused with the setter. Also the DESTDIR change seem accidental, right?

@evelikov
Copy link
Collaborator

For a little more context: the kmod tools statically link libkmod, so in practise the binary includes all the build-time values.

Thus isn't not too intuitive (IMHO) that you need libkmod.pc, when using kmod since it does not depend on the library itself. Any library users, interested in the build-time modules dir can kmod_new(), without overriding the path and read back with kmod_get_dirname() and follow-up as needed.

@hramrach
Copy link
Contributor Author

People depending on kmod.pc would be mainly people using a Linux kernel patch that was not merged. For compatibility with that the patch that actually removes kmod.pc is separate so that they can not apply/revert the patch and get the new feature at the same time.

If there is a problem with the option name feel free to suggest a better one.

There are 'getter' options, at least version and help.

Additional references would be #100 and https://bugs.gentoo.org/926431 (I did not find corresponding one on github).

@hramrach
Copy link
Contributor Author

Yes, you can probably detect some of these by compiling C tests, that's not necessarily a good interface for knowing which features are compiled into libkmod.

@hramrach
Copy link
Contributor Author

Yes, the libkmod being linked statically is a good point, while kmod links to some libraries dynamically libkmod is not one of them so all of these values are independent between kmod and libkmod.

@hramrach
Copy link
Contributor Author

Nonetheless, since e4130a4 kmod.pc is not fully usable.

Since this change was accepted, and the revert of it was not included I think the file should be dropped, anyway.

@lucasdemarchi
Copy link
Contributor

Let's leave this open as an option if the kernel patch doesn't work for just using pkg-config.

@evelikov
Copy link
Collaborator

Out of curiosity: can anyone link the ML discussion, I looked at linux-modules + linux-kbuild yet couldn't find it.

@hramrach
Copy link
Contributor Author

At this point kmod.pc is broken because it's installed in the wrong location.

Either the move should be reverted or the file removed.

Given that the move was submitted, merged, the revert not merged my impression is that the kmod.pc file is too difficult to understand and should be replaced.

@hramrach
Copy link
Contributor Author

Out of curiosity: can anyone link the ML discussion, I looked at linux-modules + linux-kbuild yet couldn't find it.

Which discussion?

@hramrach
Copy link
Contributor Author

Any decision on how to resolve the kmod.pc bad location?

@lucasdemarchi
Copy link
Contributor

@hramrach I have a patch to the kernel that I have to clean up and send to actually use it where it is and don't see a use case in which this is important. Could you clarify with a practical example?

@lucasdemarchi
Copy link
Contributor

Technically it's possible to have kmod and libkmod with mismatched
values due to them being built separately. Also on a binary OS
distribution the libkmod.pc would be installed only with libkmod
development headers, not the tool.

this can actually be solved in any of these ways: a) copy the value in libkmod.pc or b) install kmod.pc regardless since it also contains other "project information". No need to burn the castle down.

@hramrach
Copy link
Contributor Author

Also this file was initially introduced, then moved to a different
loaction because its purpose is misunderstood. Using it is awkward
because the binary search path and pkgconfig path must be matched in
case there is a possibility that multiple builds of kmod exist.

As to why multiple build of kmod might exist on a system

  • custom test build
  • build of kbuild that matches some target system which has different
    settings for sysconfdit, distconfdir, or module_directory
  • per-architecture tools on 'multiarch' system that is set up for
    building binaries/boot images for arbitrary architectures
    • when these use a filesystem scheme in which per-architecture
      libraries and modules are stored in per-architecture directories
      Linux sticks out like a sore thumb with all modules in /lib

@lucasdemarchi
Copy link
Contributor

in all those cases, you should have a pkg-config that matches the kmod/libkmod you want to use. If you are passing pkg-config ... kmod or pkg-config ... libkmod is not very important, but as I said we we can have the same info in both places (although possible, it'd be very wrong to have a config for depmod not matching the lib one).

@hramrach
Copy link
Contributor Author

The complaint about jq was that it's an additional tool needed for building the kernel.

That would be the same with libkmod, it was not need until now, only depmod was needed.

Using pkgconfig means that the pkgconfig path and binary path can be mismatched pointing to a different build of kmod. That becomes even worse when libkmod.pc and kmod.pc are in different locations.

Also alternative implementations might have hard time providing a pkgconfig file while program arguments are something that they have to implement for their version of depmod to work.

Using jq or pkgconfig in the first place is the result of overenginnering this feature. Since using mismatched libkmod and kmod is considered 'very wrong' any additional informational variables can be added to libkmod.pc. With that only one variable remains that is really needed for making use of depmod, and that can be handled by an additional program argument as done in this PR.

It has been pointed out that as implemented the short option name is easy to confuse. AFAICT the option parsing logic does need a short option name, and it cannot be set to something like ℳ to make it unlikely to be typed by the user. If you have better alternatives for short name please share. Also note that -m and -M take different number of arguments so while easy to confuse they are not easy to replace in the depmod invocation.

sysconfdir, distconfdir, module_compressions, and module_signatures are
libkmod features, these should be in libkmod.pc, not kmod.pc.

module_directory is special: While libkmod has a default for when the
directory is not specified depmod does spefcify it, and has its own
embetted value.

Technically it's possible to have kmod and libkmod with mismatched
values due to them being built separately. Also on a binary OS
distribution the libkmod.pc would be installed only with libkmod
development headers, not the tool.

Signed-off-by: Michal Suchanek <[email protected]>
This is the only value left in kmod.pc.

It is an overkill to create a pkgconfig file for one value.

Also this file was initially introduced, then moved to a different
loaction because its purpose is misunderstood. Using it is awkward
because the binary search path and pkgconfig path must be matched in
case there is a possibility that multiple builds of kmod exist.

As to why multiple build of kmod might exist on a system

 - custom test build
 - build of kbuild that matches some target system which has different
   settings for sysconfdit, distconfdir, or module_directory
 - per-architecture tools on 'multiarch' system that is set up for
   building binaries/boot images for arbitrary architectures
   - when these use a filesystem scheme in which per-architecture
     libraries and modules are stored in per-architecture directories
     Linux sticks out like a sore thumb with all modules in /lib

While in the past it was fine to have One Kmod To Rule Them All because
everything was hardcoded to a fixed value (and all compression methods
and signature options could be built at the same time) this is no longer
the case.

Values for sysconfdir, distconfdir, and module_directory may differ
between systems, and thus a build can be hosted on a system with one set
of values but targeted for a system with different set of values.

The -m option is provided to make it possible to override
module_directory.

However, for the modules to be installed in the correct location by
kernel build the kernel build needs a way to determine the default value
of module_directory, which this -M option provides.

With a corresponding change to the Linux kbuild system installing
modules should just work for both host with arbitrary module_directory
and target for which correctly configured target-depmod is available and
used during build.

There is always the option to pass arbitrary value as argument to the -m
option to build modules for a target system that uses different
module_directory, or to move the installed modules after the fact.
However, that requires somehow examining the target system out-of-band
to determine the correct value, and hardcode such value in a build
script. When working with multiple different targets regularly using a
target-specific depmod is a way to manage the depmod build-time settings
in a standardized way that works equally for other tools.

Signed-off-by: Michal Suchanek <[email protected]>
There is only one remaining variable which can be printed with a depmod
option, remove the file.

Signed-off-by: Michal Suchanek <[email protected]>
@lucasdemarchi
Copy link
Contributor

Let me try to be clear this time: I'm still very against this option. If you have a mismatching pkg-config you have to fix that problem.

Until now we never required depmod from the target and will not require it because of this path.

@hramrach
Copy link
Contributor Author

It's not required to have target-specific kmod. However, as already said, it is a way to manage target-specific settings when building for many targets aligned with how other target-specific tools work. If we don't make it possible to have target-specific kmod then kmod is that one special tool that needs to be handled differently, and with current situation of kmod.pc being 'platform independent data' that is the case.

@lucasdemarchi
Copy link
Contributor

It's not required to have target-specific kmod.

So why are you adding an option to depmod to print the path if you are not going to have a target-specific depmod?

If we don't make it possible to have target-specific kmod then kmod is that one special tool that needs to be handled differently, and with current situation of kmod.pc being 'platform independent data' that is the case.

We already make it possible via pkg-config. It's platform-independent in the sense that it corresponds to the depmod you have installed on your system - it doesn't depend on the platform. If you have other depmod installed on your system, then you need to point pkg-config to the right tree where that is coming from.

You still didn't provide a concrete use case for it. I know what the alternatives are and know of their own issues. What are you trying to support? What distro or build tools (buildroot? yocto? android?...).

As to why multiple build of kmod might exist on a system

custom test build

build of kbuild that matches some target system which has different
settings for sysconfdit, distconfdir, or module_directory

per-architecture tools on 'multiarch' system that is set up for
building binaries/boot images for arbitrary architectures
when these use a filesystem scheme in which per-architecture
libraries and modules are stored in per-architecture directories
Linux sticks out like a sore thumb with all modules in /lib

Requiring depmod on each of those doesn't make it easier to support. If you are mixing multiple depmod on the same system, you rather have a broken system. If you are providing it for building separate distros/configuration/arch, then you should point it to the pkg-config of that distro/configuration/arch rather than pointing to the depmod of that distro/configuration/arch..... PKG_CONFIG_SYSROOT_DIR / PKG_CONFIG_PATH / PKG_CONFIG_LIBDIR are your friends for this use case.

@hramrach
Copy link
Contributor Author

If you are providing it for building separate distros/configuration/arch, then you should point it to the pkg-config of that distro/configuration/arch rather than pointing to the depmod of that distro/configuration/arch..... PKG_CONFIG_SYSROOT_DIR / PKG_CONFIG_PATH / PKG_CONFIG_LIBDIR are your friends for this use case.

but kmod.pc is 'architecture independent data', can't do.

And exactly because people can't get that I want to get rid of it.

It's not worth explaining 100 times that it's broken.

@lucasdemarchi
Copy link
Contributor

but kmod.pc is 'architecture independent data', can't do.

I already said replicating that information in libkmod.pc is fine.

And exactly because people can't get that I want to get rid of it.

It's not worth explaining 100 times that it's broken.

Maybe because you keep explaining what was not asked for. I still don't know what distro / tools / concrete-example you are trying to support even if I asked for it multiple times.

@hramrach
Copy link
Contributor Author

I already said replicating that information in libkmod.pc is fine.

It's not fine. libkmod.pc was not required before.

I still don't know what distro / tools / concrete-example you are trying to support even if I asked for it multiple times.

Suppose a situation that you have libraries/modules installed in /usr/lib-$target, and binaries /usr/bin/$target-$tool.

With kmod.pc as 'architecture independent data' there is no way to support this.

@hramrach
Copy link
Contributor Author

Also if you rely on libkmod for this information then if busybox wants to support this it needs to provide libkmod.pc when it does not provide a libkmod.

The kernel would need to pick which file it uses even if the MODULE_DIRECTORY is provided by both kmod.pc and libkmod.pc.

@hramrach
Copy link
Contributor Author

ping?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants