Avoid a crash when there is a timeout when shutting down the Dask cluster

[bouweandela]

Description

Avoid a crash when there is a timeout when shutting down the Dask cluster. I have seen this happen on Levante at DKRZ recently. It does not appear to be a real problem as the cluster is shut down after the tool completes running.

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.66%. Comparing base (a328578) to head (be726ed).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2580   +/-   ##
=======================================
  Coverage   94.66%   94.66%           
=======================================
  Files         251      251           
  Lines       14287    14290    +3     
=======================================
+ Hits        13525    13528    +3     
  Misses        762      762           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[valeriupredoi]

LGTM bud, maybe tell the user what to do when the Cluster fails to shut down? ie don't panic, the process will be terminated by the host etc

[bouweandela]

How about "Timeout while trying to shut down the cluster at %s, you may want to check it was stopped."?