Function repo plugin system

[jesper-friis]

The current implementation of add_function() is very simple. It just stores the function in an internal dict. This has several drawbacks:

• it is not persistent
• it is not shared between processes or sessions (which might be needed with OTEAPI services running on a distributed system)
• it stores information outside the triplestore

I can think about two solutions:

1. Pickle the function and store the pickle in the triplestore. The pickle module in the standard library will not work, since it requires the function to be available when unpickled. But dill will work for most functions, except for functions with recursive inner functions. However, there is an article about how to also solve that case, see https://medium.com/@greyboi/serialising-all-the-functions-in-python-cd880a63b591.

2. Identify the function with a set of string properties:

   • PyPI package name (a string in the same form as requirements.txt files including version requirements), optional
   • package name, optional
   • module name
   • function name

   These can easily be stored in the triplestore. To get the conversion function, you can give module and package names as arguments to importlib.import_module() and then fetch the function with getattr() on the return module object. The PyPI package name may be useful if we want to hint the user about a missing package or to dynamically install it if it is missing. Automatic installation raises some security concerns, so if implemented, this feature should probably be explicit enabled.

   This approach has already been implemented in the convert strategy in oteapi-dlite and seems to work well. See https://github.com/EMMC-ASBL/oteapi-dlite/blob/convert-function-strategy/oteapi_dlite/strategies/convert.py#L57.

Personally I prefer the second option, since it is simple and clean and stores no byte code that might result in portability issues.