Pass coprocess parameter to WorkQueue

[benclifford]

This allows Work Queue coprocesses to be enabled from the bps yaml config without having to fiddle with Python configuration.

[jchiang87]

Looks ok to me. Can you comment here on what the coprocess functionality in work_queue provides?

[benclifford]

@jchiang87 in Work Queue by default a new Python process is started up to do the per-task Python code (because all parsl tasks are at heart Python tasks) - in the DESC case most usually this is monitoring wrapping code wrapped around a fork of the actual user process. The HTEX and thread executors do not start up a Python process per-task this way because they are already inside Python: the htex workers are implemented in Python, and the thread executor runs stuff directly inside the submitting Python process (as a thread).

The Work Queue coprocess mode gives Work Queue persistent Python workers that are used for a sequence of tasks - so those workers look more like HTEX Python workers.

In the DESC case, where I've found this helpful is in an environment where the cost of Python process startup and import is expensive: this cost gets paid once per worker, rather than once per task. That process startup and import has been expensive sometimes when working with Python coming from heavily loaded shared filesystems when there is a lot to import.