Default service account and pod security policies

[scottyhq]

An issue with the current service account and role configuration of dask-kubernetes (0.10) on a multi-user platform like jupyterhub (1.0) is that authenticated users can interact with the resources of other users and create new pods/containers of their choosing. These systems typically have authentication and users are trustworthy, but it's desirable to also run on open platforms like binderhub. It would be great to limit jupyterhub users to 1) interact with only their pod and dask worker pods that are launched by them and 2) maybe limit the containers that can be run on those pods.

Not sure what the best approach is to address this problem. Perhaps it is just a matter of configuring a pod security policy https://kubernetes.io/docs/concepts/policy/pod-security-policy/# on the cluster, or maybe some settings at the jupyterhub level (users get their own namespace?) could remedy the current situation?

pinging @jacobtomlinson @yuvipanda @jhamman and @consideRatio

[jacobtomlinson]

Thanks for raising this @scottyhq. I think this is likely going to be more of a configuration thing at the JupyterHub level rather than changes to dask-kubernetes, but there may be some things we can do to make life easier so let's track this here. We may also want to make changes in the kube spawner.

My recommendation on an approach to this would be for each user in JupyterHub to get their own namespace and service account which allows them to only manage pods within that namespace.

You should also limit the type of pods that can be created using the security policies you linked to, specifically not allowing privileged execution, limiting the pod user to jovyan and avoiding host filesystem mounting would be important.

I seem to remember there was some discussion of nested namespaces within the Kubernetes community. I'm not sure whether that went anywhere, @yuvipanda may know more. This would be great as you could have a high level namespace of pangeo-binder-staging for example and then have nested user namespaces within that. For now we could just use some naming convention to group things together.

[jacobtomlinson]

The classic KubeCluster was removed in #890. All users will need to migrate to the Dask Operator. Closing.