refactor generation of required environment variables in module files + deprecate make_module_req_guess method in EasyBlock class

[lexming]

Goal is to lay groundwork to be able to fix #3331 and add new option to control which search path variables are added in modules by simplifying the code injecting environment variables to module files.

This PR should not alter the resulting module files to what is currently generated by EB. If it changes the list of environment variables it should have no effect (e.g. because those paths are empty).

Simplification of logic handling environment variables for module files:

1. replace obscure rules defining which search paths require populated dirs with:
   • all modules require search paths that point to existing files or non-empty dirs
     (fixes directories that don't contain any library files shouldn't be added to $LD_LIBRARY_PATH #3504)
   • fake modules can have search paths to empty dirs
2. replace obscure rules defining which search paths require files in their top level directory with:
   • by default all search paths to directories can be populated at any level (recursive check)
   • this setting is now controlled by top_level_file attribute of ModuleEnvironmentVariable
3. replace convoluted rules to handle symlinked lib64 that generate duplicate paths and need explicit exceptions with:
   • paths to lib64 are always ignored if lib64 is a symlink to lib
   • paths to lib are always ignored if lib is a symlink to lib64
   • remove lib32 from list of library search paths
4. allow simple updates of environment variables in child EasyBlocks, replacing the clunky:

   def make_module_req_guess(self):
       guesses = super(EB_Clang, self).make_module_req_guess()
       guesses.update({'SOME_VAR': ['some/path]})
       return guesses
   

   with an elegant:

   self.module_load_environment.SOME_VAR = ['some/path']
   

Changelog:

• add ModuleEnvironmentVariable pseudo-dataclass to easybuild.tools.modules to hold definitions of environment variables for modules
• add ModuleLoadEnvironment singleton class to easybuild.tools.modules to hold environment definition for modules on load
• deprecate EasyBlock.make_module_req_guess in favor of ModuleLoadEnvironment
• add LibSymlink enum to easybuild.framework.easyblock to define possible linkage states of lib directories
• add install_lib_symlink attribute to EasyBlcok to keep track of symlink status between lib dirs
• add globals to easybuild.tools.config that define known directories for binaries, libraries and headers
• avoid changing the current working directory of the main process in easyblock.make_module_req by working with absolute paths instead

[lexming]

@boegel look all the beautiful green tests!

Compared to what I showed on the meeting yesterday, I have undone all changes to make_module_req_guess. So existing easyblocks continue to work without any further changes.

This seems ready on my side. I tested it with several easyblocks that add custom stuff to make_module_req_guess and I have hit no issues so far. Even things like root.py easyblock that set PYTHONPATH in make_module_req_guess instead of the more common make_module_extra do work. If you have anything in mind that I should test in particular let me know.

[boegel]

@lexming Can you look into fixing the merge conflict, and get the test to pass?

[boegel]

@lexming All remarks fixed in lexming#6

[boegel]

@lexming Shouldn't we at least log a warning or something if any entries in self.module_load_environment do not have is_path is True?

Because those cases will essentially be totally silently ignores currently?

[lexming]

Yep we can log that, not sure if this deserves a warning though. I guess there might be cases where no path is added to the module file that are ok, like a ModuleRC.

[boegel]

My suggestion relates more to entries in self.module_load_environment for which is_path is False, those cases should be exposed clearly rather than silently ignoring them?

[boegel]

@lexming Is this only done to bypass the check in expand_module_search_path that ensures that this directory actually has files in them?

Why? Is it only to get some marginal speedup?

This may be important, the "fake" module is generated right before the sanity check is run, so if for some weird reason having additional paths to empty directories in environment variables causes trouble (or worse, somehow makes the sanity check pass), it sort of invalidates the sanity check...

Is there another reason to make this exception beyond a small performance optimization?

[lexming]

At some point this was needed, but I lost the specifics. This PR increases the amount of paths that are checked for existence before injection into the module file and I remember having issues with fake modules lacking paths that are non-existent at the time of creation of the fake, but exist in the end. However, I'm not hitting those issues anymore, so I'll revert it and see if they happen again.

[boegel]

@lexming What was the trigger for the change you made in 13f735c?

That broke the test_module_environment_variable test, and for good reason I'd say...

Can you enhance the tests to cover the case that made you make the change in 13f735c?

[lexming]

@boegel actually the tests is what saved us, they properly failed. That change in 13f735c was not needed, args are packed in a list and they need unpacking.