[24.0] Always discard session after __handle_waiting_jobs is done

[mvdbeek]

This should guarantee recovery of the job monitor thread and fix #17893. Alternative to #17908, which is close but I think this avoids guessing which exceptions we gotta do this for, and since it's part of the normal control flow we have some test coverage here.

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[jdavcs]

Let's try this!

[github-actions]

This PR was merged without a "kind/" label, please correct.