Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse cwl-tes logs in real-time #56

Closed
uniqueg opened this issue Oct 9, 2018 · 1 comment
Closed

Parse cwl-tes logs in real-time #56

uniqueg opened this issue Oct 9, 2018 · 1 comment
Assignees

Comments

@uniqueg
Copy link
Member

uniqueg commented Oct 9, 2018

Is your feature request related to a problem? Please describe.
Currently logs (STDOUT/STDERR) of cwl-tes are parsed only after job completion, causing TES task IDs to be inaccessible until after a run completed. This has several severe disadvantages, such as:

  • User are unable to track the progress of individual tasks; the task_log array field in the run_log object remains empty
  • Ignorance of task IDs can lead to "zombie" runs that stay indefinitely in incomplete states, such as UNKNOWN or RUNNING
  • Cancelling a workflow run cannot be propagated to the task level, leading to a waste of resources

Describe the solution you'd like
Parse cwl-tes run logs in real-time, publish task state changes to the broker and handle them in the event listener/manager.

Describe alternatives you've considered
Parsing logs and polling TES status changes (as done by cwl-tes) are temporary solutions only. See #57 and #58 for more viable solutions with regard to polling and log parsing, respectively.

Additional context
N/A

@uniqueg uniqueg self-assigned this Oct 9, 2018
@uniqueg uniqueg added this to the Release candidate v1.0.0-rc milestone Oct 9, 2018
@uniqueg
Copy link
Member Author

uniqueg commented Oct 11, 2018

Closed by #67

@uniqueg uniqueg closed this as completed Oct 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant