Skip to content

Commit

Permalink
Redecorate apps for all executors, to get task label in monitoring
Browse files Browse the repository at this point in the history
Prior to this PR, task names appeared in the monitoring database when uisng
work queue executor because a run_command-like app was created for every
invocation, and that app was customised to have the bps task label attached.

With other executors, that code path was not used, and so other executors
default to using the parsl default of name of the underlying Python function
which in the gen3_workflow case was `run_command`.

This PR switches to making a new app for every task invocation for all
executor types, and applies the same task label modification. This makes the
task label available in monitoring with all executor types.
  • Loading branch information
benclifford committed Nov 27, 2023
1 parent 9ea5a6d commit 25e8a74
Showing 1 changed file with 11 additions and 12 deletions.
23 changes: 11 additions & 12 deletions python/desc/gen3_workflow/parsl_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,17 +81,10 @@ def start_pipeline(config_file, outfile=None, mode='symlink'):
_EXEC_DONE = 'exec_done'


def run_command(command_line, inputs=(), stdout=None, stderr=None):
"""
Run a command line as a parsl.bash_app.
"""
return command_line


RUN_COMMANDS = dict(small=small_bash_app(run_command),
medium=medium_bash_app(run_command),
large=large_bash_app(run_command),
local=local_bash_app(run_command))
RUN_DECORATORS = dict(small=small_bash_app,
medium=medium_bash_app,
large=large_bash_app,
local=local_bash_app)


class ResourceSpecs:
Expand Down Expand Up @@ -182,7 +175,13 @@ def wq_run_command(command_line, inputs=(), stdout=None, stderr=None,
except AttributeError:
# Using executors that don't have a mem_per_worker attribute.
pass
return RUN_COMMANDS[job_size]

def job_run_command(command_line, inputs=(), stdout=None, stderr=None):
return command_line

job_run_command.__name__ = task_label

return RUN_DECORATORS[job_size](job_run_command)


@parsl.python_app(executors=['submit-node'])
Expand Down

0 comments on commit 25e8a74

Please sign in to comment.