routing to dashboard with deploy_mode="remote"

[Techn0logic]

It is fantastic that dask scheduler now starts as a standalone pod instead of as a local process. (>=v0.10.0)

However, now can't use jupyter-server-proxy to easily route to the dask dashboard.

As a dask-kubernetes user I would like to deploy and scale KubeCluster and be able to monitor jobs with the dask-labextension plugin. Would welcome any ideas on how to achieve that. - thanks

[jacobtomlinson]

Thanks for raising this!

I am aware of this problem, this is actually why the remote scheduler is not the default option. You can find some more discussion and reasoning in #162 about this.

There is also a PR here jupyterhub/jupyter-server-proxy#154 which would add the required functionality to jupyter-server-proxy, however it is currently blocked on security concerns.

[bolliger32]

Just starting to try out this functionality as well. Will be very helpful when we're able to stably deploy a scheduler on a remote pod. Looks like #154 was merged and released. I've been playing around but have not been able to figure out how to load the dashboard from a remote scheduler. I'm having a hard time telling if the jupyter-server-proxy functionality update should directly enable viewing of a remote scheduler via the dashboard within dask_kubernetes (with simply the appropriate configs) or whether some actual development is needed. Any direction would be much appreciated - this will be super valuable functionality once we can get it up and running

[jacobtomlinson]

Now that #154 is implemented we can add the functionality in here. Things that need to happen:

• Minimum version of the proxy needs updating
• Dashboard url needs updating to use the new proxy format
• Dashboard service domain needs to dynamically added to the proxy whitelist

[jacobtomlinson]

The classic KubeCluster was removed in #890. All users will need to migrate to the Dask Operator. Closing.