Fix Cluster Autoscaler Evictions

[frctnlss]

The Kubernetes Cluster Autoscaler evicts pods on scale-down events based on a number of criteria. This PR makes the default behavior of new work pools, based on the Kubernetes worker, to prevent eviction.

The option to disable this setting is added for situations like GKE Autopilot which has a max number of running pods with the required annotation. This also means the backoff limit must be increased to accommodate restarts. Prefect will otherwise mark the Flow as Crashed when the backoff limit is reached.

Checklist

• References any related issue by including "Closes #" or "Closes ".
  • If no issue exists and your change is not a small fix, please create an issue first.
• Includes tests or only affects documentation.
• Passes pre-commit checks.
  • Run pre-commit install && pre-commit run --all locally for formatting and linting.
• Includes screenshots of documentation updates.
  • Run mkdocs serve view documentation locally.
• Summarizes PR's changes in CHANGELOG.md

[frctnlss]

This line separation was completed due to conflicting rules between black and flake8. Reference the docs here. I am open to alternative options, but the options I saw were: the line break, increasing the line length arbitrarily, or ignoring the E501 rule.

[desertaxle]

Feel free to add a comment to ignore the rule!

[desertaxle]

Thanks for the PR @frctnlss! I would prefer to set the cluster-autoscaler.kubernetes.io/safe-to-evict to False by default and leave out the new safe_to_evict and backoff_limit variables. Those values will still be editable, but I don't think they should be default variables, especially because increasing the backoff limit can have strange side-effects when used with Prefect retries.

[desertaxle]

Closing this PR because it appears to be stale. Feel free to reopen and make the requested changes if this functionality is still needed.