Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[13pt] Check and fix all multi proc code to include the 'futures' MP system #306

Open
RobHanna-NOAA opened this issue Feb 29, 2024 · 0 comments
Labels
bug Something isn't working low priority

Comments

@RobHanna-NOAA
Copy link
Collaborator

RobHanna-NOAA commented Feb 29, 2024

All MP's children functions have the ability to detect if there is a problem and shut down's its own process. However, we need to use the future system which can detect if the MP child was aborted and also have a result passed back from the child MP process. You can then use that to update any lists that were feed into the Pool the first place.

Two problems are triggered without this fix.

  1. A list feed into a multi proc will need to be updated to remove failed child MP, especially if that list continues to used in more processing.
  2. It is possible for all MP children to fail. If all fail, we need to shut the whole app down.

PS. we already have some examples where we do use the 'futures' system. We just need to check if they need to update any lists for failed MP children.

Update: Mar 1 - Put this on hold.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working low priority
Projects
None yet
Development

No branches or pull requests

1 participant