chore: update documentation for Amalthea (#2319)

Co-authored-by: Pamela <[email protected]>
Co-authored-by: Andreas Bleuler <[email protected]>

README.rst

@@ -121,9 +121,13 @@ The Renku project consists of several sub-repositories:
  Knowledge Graph services
 
 - `renku-notebooks <https://github.com/SwissDataScienceCenter/renku-notebooks>`_:
- a lightweight service for handling interactive notebooks through JupyterHub
+ a lightweight service for handling interactive notebooks through Jupyter servers
+ and a k8s operator
 
-- `renku-jupyter <https://github.com/SwissDataScienceCenter/renku-jupyter>`_:
+- `amalthea <https://github.com/SwissDataScienceCenter/amalthea>`_: k8s operator for
+ Jupyter servers.
+
+- `renkulab-docker <https://github.com/SwissDataScienceCenter/renkulab-docker>`_:
  base images for interactive sessions
 
 - `renku-python <https://github.com/SwissDataScienceCenter/renku-python>`_:

docs/_static/graphviz/notebook_service_architecture.dot

@@ -6,28 +6,30 @@ strict digraph architecture {
  node [shape="rect", style="filled,rounded", fontname="Raleway"];
  edge [fontname="Raleway"]
 
+
  GitLab [fillcolor="lightblue"]
- storage [fillcolor="lightblue", label="object store", shape="cylinder"]
-
- JupyterHub [fillcolor="lightblue"]
+ Amalthea [fillcolor="lightblue"]
  "notebook-service" [fillcolor="#f4d142" label="Notebook service"]
- notebook [label="Jupyter\nnotebook", shape="rect", fillcolor="#f4d142"]
+ notebook [label="Jupyter\nserver", shape="rect", fillcolor="#f4d142"]
+ oauth2proxy [label="Oauth2\nProxy", shape="rect", fillcolor="#f4d142"]
+ gitproxy [label="Git\nProxy", shape="rect", fillcolor="#f4d142"]
  "init-container" [shape="rect", fillcolor="#f4d142"]
+
 
- "notebook-service" -> JupyterHub [label=" API"]
+ "notebook-service" -> Amalthea [label=" API"]
 
  subgraph cluster_notebook {
- label="notebook pod"
+ label="JupyterServer custom resource"
  style="dashed";
- "init-container"
  notebook
- notebook -> "init-container" [label=" mount volume"]
- {rank=same; "init-container", notebook}
+ oauth2proxy
+ "init-container"
+ {rank=same; "init-container", notebook, oauth2proxy, gitproxy}
+
  }
 
  "notebook-service" -> GitLab [label=" repo permissions\n image build status"]
- JupyterHub -> notebook [label=" spawn", lhead=cluster_notebook]
- "notebook" -> GitLab [label=" git pull/push"]
+ Amalthea -> notebook [label=" spawn", lhead=cluster_notebook]
+ gitproxy -> GitLab [label=" git pull/push"]
  "init-container" -> "GitLab" [label=" git clone"]
- notebook -> storage [label=" fetch image\n push/pull LFS"]
 }

docs/admin/configurations/standalone-gitlab.rst

@@ -28,23 +28,3 @@ Scopes:
  openid (Authenticate using OpenID Connect)
 
 Copy the secret and client ID and paste it to the dedicated places in renku-values.yaml (step 3)
-
-- Jupyter notebooks.
-
-Name: jupyter
-
-Callback URLs:
-
-.. code-block:: console
-
- https://<your-renku-dns>/jupyterhub/hub/oauth_callback
- https://<your-renku-dns>/jupyterhub/hub/api/oauth2/authorize
-
-Scopes:
-
-.. code-block:: console
-
- api (Access the authenticated user's API)
- read_user (Read the authenticated user's personal information)
-
-Copy the secret and client ID and paste it to the dedicated places in renku-values.yaml (step 3)

docs/developer/example-configurations/gitlab.rst

@@ -7,7 +7,7 @@ GitLab is a component of the Renku platform. Running it is quite resource-intens
 For development purposes or if you already have an existing GitLab instance running,
 you might want to use this instance as a backend service for Renku.
 
-In the section about :ref:`running Renku locally <running_on_minikube>` we give a
+In the section about :ref:`running Renku locally <running_on_kind>` we give a
 detailed description of how to bring up a local version of the platform using `<gitlab.com>`_
 while also using ``gitlab.com`` as an identity provider. Note that in Renku deployments
 which include their own GitLab, the dependency is the other way around, meaning

docs/developer/general/json_web_tokens.rst

@@ -47,4 +47,4 @@ log-in process. While there is currently no Renku backend service which uses the
 Keycloak access token for authorizing access to its resources, this is likely
 to change in the future.
 
-Note that the access tokens issued by JupyterHub and GitLab are **not** JWTs.
+Note that the access tokens issued by GitLab are **not** JWTs.

docs/developer/services/api_gateway.rst

@@ -4,7 +4,7 @@ API gateway
 ===========
 
 The API gateway acts as a middle layer between the Renku clients and the
-different backend services (GitLab, JupyterHub, Renku storage service, ...).
+different backend services (GitLab, Renku notebooks, Renku core service, ...).
 More specifically, the API gateway makes life easier for client developers in
 the following ways:
 
@@ -15,9 +15,9 @@ the following ways:
  all clients.
 
 - **Resource specific routing**:
- Notebook servers are stored in JupyterHub, project details in GitLab and datasets
- have to be fetched through the storage service - this must not be the worry of
- each client (developer), instead it is handled by the gateway.
+ Notebook servers are stored in custom k8s resources created by Amalthea.
+ Project details in GitLab and datasets have to be fetched through the storage service 
+ - this must not be the worry of each client (developer), instead it is handled by the gateway.
 
 - **Token handling**:
  Even with all backend services offering the possibility to rely on an OpenID-connect

docs/developer/services/notebooks_service.rst

@@ -6,21 +6,25 @@ Notebooks service
 The notebooks service provides an interactive computing environment for every
 commit in a project's history to each user that has sufficient access rights.
 
-JupyterHub integration
+Amalthea integration
 ----------------------
 
-`JupyterHub <https://jupyterhub.readthedocs.io/en/stable/>`_ is a multi-user
-server for spawning interactive `Jupyter notebooks <https://jupyter-
-notebook.readthedocs.io/en/stable/>`_. Renku uses JupyterHub to manage
+`Amalthea <https://github.com/SwissDataScienceCenter/amalthea>`_ is a
+k8s operator for spawning interactive `Jupyter notebooks <https://jupyter-
+notebook.readthedocs.io/en/stable/>`_. Renku uses Amalthea to manage
 sessions and the ``notebooks-service`` extends the standard
-JupyterHub functionality by providing tight integration with GitLab.
+Amalthea functionality by providing tight integration with GitLab.
 
-The notebooks are provided by the JupyterHub server. A new "named" server is
+The notebooks are provided by the Jupyter Server. A new "named" server is
 spawned for every unique request. A notebook server launch is initiated by
-accessing the
-``<PLATFORM_URL>/<JUPYTERHUB_PREFIX>/services/notebooks/<namespace>/<project-
-name>/<commit-sha>`` URL. This means that two users collaborating on a project
-use the same URL, but each will receive their own notebook server.
+posting a request to the ``<PLATFORM_URL>/api/notebooks/servers`` URL.
+The request needs to contain information about the project, commit, Docker image
+and resources that are required by the specific server. 
+In addition, if two or more users collaborating on a project
+use the same URL with the same data passed in the request, 
+each will receive their own notebook server.
+
+https://dev.renku.ch/
 
 By default, a Renku project will include a ``.gitlab-ci.yml`` file that
 contains an ``image_build`` stage which creates an image for every push (see the

docs/developer/services/services_architecture.rst

@@ -28,14 +28,14 @@ Renku has the following custom components:
 - renku_: the meta repository with deployment scripts, documentation and kubernetes helm charts
 - :ref:`renku-gateway <api_gateway>`: an API gateway connecting clients to the APIs of the different backend services
 - :ref:`renku-graph <graph_services>`: a collection of services concerned with activating, building and querying the Renku knowledge graph
-- :ref:`renku-notebooks <notebooks_service>`: a service integrating GitLab repositories with JupyterHub
+- :ref:`renku-notebooks <notebooks_service>`: a service integrating GitLab repositories with Amalthea and Jupyter servers
 - renku-python_: python API & Command Line Interface (CLI)
 - renku-ui_: web front-end interface
 
 
 In addition, we make use of:
 
-- JupyterHub_: management of interactive notebook servers
+- Amalthea_: k8s operator used to launch Jupyter servers
 - GitLab_: repository management, CI and various related APIs
 - Keycloak_: identity management and user authentication
 
@@ -66,6 +66,6 @@ familiarized with a few foundational concepts:
 .. _renku: https://github.com/SwissDataScienceCenter/renku
 .. _renku-python: https://github.com/SwissDataScienceCenter/renku-python
 .. _renku-ui: https://github.com/SwissDataScienceCenter/renku-ui
-.. _JupyterHub: https://github.com/jupyterhub/jupyterhub
+.. _Amalthea: https://github.com/SwissDataScienceCenter/amalthea
 .. _GitLab: https://gitlab.com/
 .. _Keycloak: https://www.keycloak.org/