Scoping build design of Turing hosted BinderHub

[sgibson91]

Summary

This issue is to be a discussion surrounding the design of Hub23 (the Turing hosted BinderHub instance).

There is a proposal for 3 levels of access:

Level 1

Any member of the public is able to use minimal Azure resources to verify their own project is reproducible. For all intents and purposes, a copy of mybinder.org. See #298

Level 2

Any member of the public is able to use Hub23 to verify Turing research is reproducible (published papers or public projects) and computational Azure resources are available for those listed projects. See #297

Level 3

Any member of the Turing Institute have the option of bringing their own Azure subscription (see #292 ). Public and private repos are supported (see #291 and #346 ). Greater computational resources are also available (see #296 ).

What needs to be done?

• Use Azure Active Directory/GitHub for authentication of Turing Institute members (see Decide how to Authenticate Users of the Turing BinderHub #290 Can't get GitHub authentication to work on an Autoscaling BinderHub #410 )
• "Scratch" Docker/Container registry that is regularly cleaned and secure for private repos (see Turing Container Registry for Binder image management #293 )
• Autoscaling/restriction of Azure resources for appropriate access levels (see Enable Autoscaling of the BinderHub cluster #300 )
• Security of access tokens/keys/passwords etc. (see Azure Key Vault for Turing BinderHub #320 )
• Add any other points that are relevant or missing to this list

Who can help?

• People with BinderHub/JupyterHub experience
• People with Azure experience
• People with Kubernetes experience

---

Updates

07/03/2019 - Service Principal granted by Turing IT team. Need to decide what sort of VM to deploy onto and what node count. See #306

07/03/2019 - Organisation Authentication working on @sgibson91 's test BinderHub. Need to do some further digging in order to convince ourselves the read:org scope is only allowing read only access to organisations, teams and memberships. See #290 #410

[KirstieJane]

OMG - I'm frantically working on a different project for a meeting this evening - but I LOVE the name HUB23 😆 🤣 😆 💯

[sgibson91]

Once I realised Turing + Binder = tinder, I had to come up with a better option! 🤣

[sgibson91]

Just want to log some useful links I've found pertaining to the BinderHub in case I end up shutting things down on my laptop for the workshop tomorrow.

---

• Script for auto-deploy to Google: https://github.com/nicain/binder-deploy

• Accessing private repos: https://binderhub.readthedocs.io/en/latest/setup-binderhub.html#accessing-private-repositories

• Customising the JupyterHub deployments: https://zero-to-jupyterhub.readthedocs.io/en/latest/extending-jupyterhub.html#customizing-your-deployment

• JupyterHub configuration reference: https://zero-to-jupyterhub.readthedocs.io/en/latest/reference.html#configuration-reference

• JupyterHub optimisations: https://zero-to-jupyterhub.readthedocs.io/en/latest/optimization.html#optimizations

• BinderHub docs PDF: https://media.readthedocs.org/pdf/binderhub/latest/binderhub.pdf

• Authentication: https://zero-to-jupyterhub.readthedocs.io/en/stable/authentication.html#authentication

[sgibson91]

It looks like the Azure AKS Kubernetes service only supports a single "agent pool", which can only have one type/size of VM. If/when we look at supporting different VM types (e.g. GPU and CPU) we could either:

• deploy one BinderHub per compute type on AKS
• Look at the Azure AKS Engine, which supports multiple agent pools per Kubernetes cluster.

[betatim]

jupyterhub/zero-to-jupyterhub-k8s#1178 (comment) for a discussion on "give different resources to different users". There it is in the context of a JupyterHub but once you have auth working for your BinderHub it should be quite similar.

[sgibson91]

Plan is that Hub23 will no longer be a BinderHub, but a JupyterHub instead!

• Moving from BinderHub to Zero-to-JupyterHub alan-turing-institute/hub23-deploy#426