Dask 2023.9.2 rise an assertion error

[dbalabka]

Describe the issue:

Try to create a cluster:

from dask_kubernetes.operator import KubeCluster

cluster = KubeCluster(
    # We assume that such a namespace exists
    name='my-cluster',
    namespace='dask-operator'
)

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
File [.../.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:238](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/.../.../.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:238), in KubeCluster.__init__(self, name, namespace, image, n_workers, resources, env, worker_command, port_forward_cluster_ip, create_mode, shutdown_on_close, idle_timeout, resource_timeout, scheduler_service_type, custom_cluster_spec, scheduler_forward_port, jupyter, loop, asynchronous, **kwargs)
    235 try:
    236     # Validate `resources` param is a dictionary whose
    237     # keys must either be 'limits' or 'requests'
--> 238     assert isinstance(
    239         self.resources, dict
    240     ), f"resources must be dict type, found {type(resources)}"
    241     for field in self.resources:

AssertionError: resources must be dict type, found <class 'NoneType'>

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)

...

File [.../.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:249](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/.../.../.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:249), in KubeCluster.__init__(self, name, namespace, image, n_workers, resources, env, worker_command, port_forward_cluster_ip, create_mode, shutdown_on_close, idle_timeout, resource_timeout, scheduler_service_type, custom_cluster_spec, scheduler_forward_port, jupyter, loop, asynchronous, **kwargs)
    247             raise ValueError(f"resources has unknown field '{field}'")
    248 except AssertionError as e:
--> 249     raise TypeError from e
    251 name = name.format(
    252     user=getpass.getuser(), uuid=str(uuid.uuid4())[:10], **os.environ
    253 )
    254 self._instances.add(self)

TypeError:

Anything else we need to know?:
We use Kubernetes on-prem:

Version: v1.24.14+rke2r1
OS: Ubuntu 20.04.5 LTS
Container Runtime: containerd://1.7.1-k3s1

Environment:

• Dask version: 2023.9.2
• Python version: 3.10
• Operating System: WSL
• Install method (conda, pip, source): poetry

dask-kubernetes = "~2023.9.0"
dask = "~2023.9.0"
s3fs = "^2023.6.0"

[dbalabka]

Adding resources param into KubeCluster constructor lead to different error:

File [~/.../.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:892](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/.../.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:892), in make_worker_spec(image, n_workers, resources, env, worker_command)
    889 if isinstance(worker_command, str):
    890     worker_command = worker_command.split(" ")
--> 892 args = worker_command + [
    893     "--name",
    894     "$(DASK_WORKER_NAME)",
    895     "--dashboard",
    896     "--dashboard-address",
    897     "8788",
    898 ]
    900 return {
    901     "replicas": n_workers,
    902     "spec": {
   (...)
    919     },
    920 }

TypeError: unsupported operand type(s) for +: 'NoneType' and 'list'

[dbalabka]

Next error:

File [~/.../.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:456](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/.../~/src/demand-forecasting/.venv/lib/python3.10/site-packages/dask_kubernetes/operator/kubecluster/kubecluster.py:456), in KubeCluster._wait_for_controller(self)
    454 start = time.time()
    455 cluster = await DaskCluster.get(self.name, namespace=self.namespace, timeout=30)
--> 456 while start + self._resource_timeout > time.time():
    457     if await cluster.ready():
    458         return

TypeError: unsupported operand type(s) for +: 'float' and 'NoneType'

Sound like missing default values for multiple input parameters.

[jacobtomlinson]

I think this is fixed by #824