Use logging from standard library

[Hugovdberg]

PyEmu implements a custom logger to log messages from various parts of the package. Python comes with a logging module that provides the same functionality, and more. Also, PstFrom is currently hardcoded to log to a file in the current working directory, as well as an optional logging to stdout. I'm using pyemu within a snakemake workflow, with log files being stored in a dedicated directory, so I would like to be able to specify how and where the messages are logged. The standard library also provides means to configure different log levels for different streams (for example debug and info messages only to file, but warnings and errors also to the screen, or even custom log handlers).

Would you guys be open to a pull request that replaces the current logger with the built-in logging interface? It would mean that some options need to be removed, such as the echo argument to various helpers. We would also need to add a small tutorial for people who would like to keep the current functionality of logging to both a file and stdout, but in return the users get more flexibility both in what is logged as well as how it is logged. This removes some (in my humble opinion ;-) ) unneccessarily opnionated implementation from library code, which makes the library more pythonic.

[jtwhite79]

sounds good to me as long as we can still get timestamps and weave the logger into the helpers? Also, we need to retain the snarky and insulting messages! haha

@briochh?

[Hugovdberg]

The nice thing about the standard logging module is that it allows you much more fine grained control over the log messages. You can keep logging all messages, from anywhere in the library, but the user of the library gets more control how and where the messages are going.

It is not necessary for the library to provide timestamps as they can be included by the handler. The default line format does not include the timestamp, but it is easily added. It might be useful to provide a helper function that sets up the logging module similar to the current format so those who want the pyemu recommended settings can simply call pyemu.set_default_logging, and those who want more control setup logging as they wish.

I'll see if I can whip up a draft pull request for the pst_from module soon, so you can see how it would work.

[Hugovdberg]

@jtwhite79 what versions of Python should be supported? The travis config suggests Python versions 2.7, 3.5, 3.6 and 3.7 are used for testing, but the appveyor config suggests only Python 3.7 is tested. Also, PstFrom._setup_dirs uses f-strings, so that will not run pre-Python 3.6. Can I assume 3.7+?

[jtwhite79]

I think 3.7+ is fine. We are really only using github actions anymore....

[briochh]

Sounds like a solid plan to me @Hugovdberg. Thanks for having a dig into this.

[briochh]

Hi @Hugovdberg, are you still interested in looking into this? There is definitely value in more consistent logging approach within the code, but things may have moved around a bit since your initial issue was raised (and we are also at python 3.8+)

[Hugovdberg]

Hi @briochh, sorry for the very, very..., late reply. I hadn't been using pyemu as much as I should have been, but I created #544 to log my work on this issue. I will look into this a little more over the coming weeks.