From fa9901a4fdf49f903131f45b61b48e99ccac69e3 Mon Sep 17 00:00:00 2001 From: antonio Date: Fri, 26 Jul 2024 10:03:54 +0200 Subject: [PATCH 1/4] Updated apricot tutorial and readme --- README.md | 79 +++++++++++++++++++++++++++++++++++--- src/apricot-tutorial.ipynb | 6 +-- 2 files changed, 77 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 7bf03df..e706cbf 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,83 @@ -# apricot +# APRICOT [![Github Actions Status](https://github.com/grycap/apricotlab/workflows/Build/badge.svg)](https://github.com/grycap/apricotlab/actions/workflows/build.yml) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/grycap/apricotlab/main?urlpath=lab) -Advanced Platform for Reproducible Infrastructure in the Cloud via Open Tools. +Advanced Platform for Reproducible Infrastructure in the Cloud via Open Tools for JupyterLab. -## Requirements -- JupyterLab >= 4.0.0 +## Introduction + +APRICOT is an open-source extension to support customised virtual infrastructure deployment and usage from Jupyter notebooks. It allows multi-cloud infrastructure provisioning using a wizard-like GUI that guides the user step by step through the deployment process. It implements IPython magic functionality to use and manage the deployed infrastructures within Jupyter notebooks for increased usability. + + +## Experiment replication methodology + +APRICOT can be used to achieve reproducible experiments for experiments that require complex customised computing infrastructures using Jupyter notebooks. The key points to develop reproducible experiments using APRICOT extensions are: + + - APRICOT provides a set of predefined configurable infrastructures to fit the experiments. Any researcher can easily deploy the same computing infrastructure than the one used in a previous experimentation carried out with the deployed infrastructure in APRICOT. + + - APRICOT allows remote execution of commands at the deployed infrastructures to ease interaction. So, extra needed software can be documented and installed at the infrastructure within the same Jupyter notebook where the experimentation has been documented in order to be executed step by step. + + - Since APRICOT extension uses Jupyter notebooks as base environment, all the experiment can be documented using text, life code and images. + + + ## Requirements + +APRICOT requires the Infrastructure Manager client to deploy the infrastructure and get the access credentials. The installation details can be found at [IM documentation](https://imdocs.readthedocs.io/en/devel/gstarted.html). + +Also, APRICOT requires a [Jupyter installation](https://jupyter.org/install), since uses its environment to run. It is compatible with JupyterLab >= 4.0.0. + + +### Components + +![Alt text](docs/images/APRICOT_components.png?raw=true "Components") + +APRICOT has been constructed using the following components: + +- [**Jupyter**](https://jupyter.org/), an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. +- [**IM**](https://www.grycap.upv.es/im/index.php), an open-source virtual infrastructure provisioning tool for multi-Clouds. + + +### Infrastructure management + +To manage and use previous deployed infrastructures within Jupyter notebook environment, a set of Ipython magic functions have been implemented. These functions are listed below: + +* Magic lines: + * **apricot_log**: + * Arguments: infrastructure identifier + * Returns: The configuration logs of specified infrastructure + * **apricot_ls**: Takes no arguments and returns a list with all the deployed infrastructures using this extension. + * **apricot_info**: + * Arguments: infrastructure identifier + * Returns: The specifications of specified infrastructure + * **apricot_vmls**: + * Arguments: infrastructure identifier. + * Return: A list of working nodes and their status at the specified infrastructure. + * **apricot_upload**: Upload specified local files into the specified infrastructure destination path. + * Arguments: infrastructure identifier, upload files paths, destination path. + * **apricot_download**: Download files located at specified infrastructure to local storage. + * Arguments: infrastructure identifier, download files paths, local destination path. + * Magic line and cell: + * **apricot**: Perform multiple tasks depending on input command. + * exec: Takes as arguments a infrastructure identifier and a command to be executed in the specified infrastructure. This call is synchronous. + * list: Same as *apricot_ls* + * destroy: Take a infrastructure identifier as argument an destroys the infrastructure. + +Like any Jupyter magics, these must be lodaded at the notebook using *%reload_ext apricot_magic* or configure jupyter to load these magics in all notebooks. + + +### Docker + +A Dockerfile has been provided to construct a docker image with Jupyter and APRICOT configured. Use + +`` docker build -t apricotlab . `` + +to build the image. Then, use + +`` docker run --publish 8888:8888 apricotlab `` + +to create and execute a container. The container will start automatically a Jupyter server with APRICOT preconfigured. Then, use the url provided by Jupyter to access to the server. -## Contributing ### Development install diff --git a/src/apricot-tutorial.ipynb b/src/apricot-tutorial.ipynb index e4fdc77..74c802d 100644 --- a/src/apricot-tutorial.ipynb +++ b/src/apricot-tutorial.ipynb @@ -4,9 +4,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### Welcome to the APRICOT extension. This plugin designed for Jupyter Notebooks allows you to deploy and manage virtual infrastructures using the Infrastructure Manager. In this notebook, we will cover the basics to help you get started with APRICOT.\n", + "#### Welcome to the APRICOT extension. This plugin designed for Jupyter Notebook allows you to deploy and manage virtual infrastructures using the [Infrastructure Manager](https://imdocs.readthedocs.io/en/devel/) tool. In this notebook, we will cover the basics to help you get started with APRICOT.\n", "\n", - "First of all, let's create create a simple infrastructure to execute the following example notebook. To do so, click in the **Deployment menu** button in the toolbar above and follow the steps to deploy your first infrastructure. Any infrastructure should be able to execute all commands.\n", + "First of all, let's create a infrastructure to execute the following example notebook. To do so, click in the **Deployment menu** button in the toolbar above and follow the steps to deploy your first infrastructure. Any infrastructure should be able to execute all commands.\n", "\n", "Once the infrastructure has been deployed, we can start playing with the notebook." ] @@ -58,7 +58,7 @@ "metadata": {}, "outputs": [], "source": [ - "infrastructure_id = \"f32cf700-353f-11ef-8d60-e687f511a619\"\n", + "infrastructure_id = \"aa624830-4b20-11ef-a01f-1abcb0121196\"\n", "vm_id = \"0\"" ] }, From af073226d4d78e900bc6124b224f3a08923712c1 Mon Sep 17 00:00:00 2001 From: antonio Date: Fri, 26 Jul 2024 10:06:59 +0200 Subject: [PATCH 2/4] fixed readme --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index e706cbf..fac0b5a 100644 --- a/README.md +++ b/README.md @@ -30,8 +30,6 @@ Also, APRICOT requires a [Jupyter installation](https://jupyter.org/install), si ### Components -![Alt text](docs/images/APRICOT_components.png?raw=true "Components") - APRICOT has been constructed using the following components: - [**Jupyter**](https://jupyter.org/), an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. From 5cec5e3cc12aeb1ee6ec54c7aa6bb77530deca18 Mon Sep 17 00:00:00 2001 From: antonio Date: Fri, 26 Jul 2024 10:08:11 +0200 Subject: [PATCH 3/4] linted readme --- README.md | 65 +++++++++++++++++++++++++------------------------------ 1 file changed, 29 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index fac0b5a..c3a0ac6 100644 --- a/README.md +++ b/README.md @@ -4,30 +4,26 @@ [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/grycap/apricotlab/main?urlpath=lab) Advanced Platform for Reproducible Infrastructure in the Cloud via Open Tools for JupyterLab. - -## Introduction +## Introduction APRICOT is an open-source extension to support customised virtual infrastructure deployment and usage from Jupyter notebooks. It allows multi-cloud infrastructure provisioning using a wizard-like GUI that guides the user step by step through the deployment process. It implements IPython magic functionality to use and manage the deployed infrastructures within Jupyter notebooks for increased usability. - ## Experiment replication methodology APRICOT can be used to achieve reproducible experiments for experiments that require complex customised computing infrastructures using Jupyter notebooks. The key points to develop reproducible experiments using APRICOT extensions are: - - APRICOT provides a set of predefined configurable infrastructures to fit the experiments. Any researcher can easily deploy the same computing infrastructure than the one used in a previous experimentation carried out with the deployed infrastructure in APRICOT. +- APRICOT provides a set of predefined configurable infrastructures to fit the experiments. Any researcher can easily deploy the same computing infrastructure than the one used in a previous experimentation carried out with the deployed infrastructure in APRICOT. - - APRICOT allows remote execution of commands at the deployed infrastructures to ease interaction. So, extra needed software can be documented and installed at the infrastructure within the same Jupyter notebook where the experimentation has been documented in order to be executed step by step. +- APRICOT allows remote execution of commands at the deployed infrastructures to ease interaction. So, extra needed software can be documented and installed at the infrastructure within the same Jupyter notebook where the experimentation has been documented in order to be executed step by step. - - Since APRICOT extension uses Jupyter notebooks as base environment, all the experiment can be documented using text, life code and images. - +- Since APRICOT extension uses Jupyter notebooks as base environment, all the experiment can be documented using text, life code and images. - ## Requirements +## Requirements APRICOT requires the Infrastructure Manager client to deploy the infrastructure and get the access credentials. The installation details can be found at [IM documentation](https://imdocs.readthedocs.io/en/devel/gstarted.html). Also, APRICOT requires a [Jupyter installation](https://jupyter.org/install), since uses its environment to run. It is compatible with JupyterLab >= 4.0.0. - ### Components APRICOT has been constructed using the following components: @@ -35,48 +31,45 @@ APRICOT has been constructed using the following components: - [**Jupyter**](https://jupyter.org/), an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. - [**IM**](https://www.grycap.upv.es/im/index.php), an open-source virtual infrastructure provisioning tool for multi-Clouds. - ### Infrastructure management To manage and use previous deployed infrastructures within Jupyter notebook environment, a set of Ipython magic functions have been implemented. These functions are listed below: -* Magic lines: - * **apricot_log**: - * Arguments: infrastructure identifier - * Returns: The configuration logs of specified infrastructure - * **apricot_ls**: Takes no arguments and returns a list with all the deployed infrastructures using this extension. - * **apricot_info**: - * Arguments: infrastructure identifier - * Returns: The specifications of specified infrastructure - * **apricot_vmls**: - * Arguments: infrastructure identifier. - * Return: A list of working nodes and their status at the specified infrastructure. - * **apricot_upload**: Upload specified local files into the specified infrastructure destination path. - * Arguments: infrastructure identifier, upload files paths, destination path. - * **apricot_download**: Download files located at specified infrastructure to local storage. - * Arguments: infrastructure identifier, download files paths, local destination path. - * Magic line and cell: - * **apricot**: Perform multiple tasks depending on input command. - * exec: Takes as arguments a infrastructure identifier and a command to be executed in the specified infrastructure. This call is synchronous. - * list: Same as *apricot_ls* - * destroy: Take a infrastructure identifier as argument an destroys the infrastructure. - -Like any Jupyter magics, these must be lodaded at the notebook using *%reload_ext apricot_magic* or configure jupyter to load these magics in all notebooks. - +- Magic lines: + - **apricot_log**: + - Arguments: infrastructure identifier + - Returns: The configuration logs of specified infrastructure + - **apricot_ls**: Takes no arguments and returns a list with all the deployed infrastructures using this extension. + - **apricot_info**: + - Arguments: infrastructure identifier + - Returns: The specifications of specified infrastructure + - **apricot_vmls**: + - Arguments: infrastructure identifier. + - Return: A list of working nodes and their status at the specified infrastructure. + - **apricot_upload**: Upload specified local files into the specified infrastructure destination path. + - Arguments: infrastructure identifier, upload files paths, destination path. + - **apricot_download**: Download files located at specified infrastructure to local storage. + - Arguments: infrastructure identifier, download files paths, local destination path. +- Magic line and cell: + - **apricot**: Perform multiple tasks depending on input command. + - exec: Takes as arguments a infrastructure identifier and a command to be executed in the specified infrastructure. This call is synchronous. + - list: Same as _apricot_ls_ + - destroy: Take a infrastructure identifier as argument an destroys the infrastructure. + +Like any Jupyter magics, these must be lodaded at the notebook using _%reload_ext apricot_magic_ or configure jupyter to load these magics in all notebooks. ### Docker A Dockerfile has been provided to construct a docker image with Jupyter and APRICOT configured. Use -`` docker build -t apricotlab . `` +`docker build -t apricotlab .` to build the image. Then, use -`` docker run --publish 8888:8888 apricotlab `` +`docker run --publish 8888:8888 apricotlab` to create and execute a container. The container will start automatically a Jupyter server with APRICOT preconfigured. Then, use the url provided by Jupyter to access to the server. - ### Development install Note: You will need NodeJS to build the extension package. From 980fd5f08393851fc7410558c9ec28504e2a539b Mon Sep 17 00:00:00 2001 From: antonio Date: Fri, 26 Jul 2024 10:14:37 +0200 Subject: [PATCH 4/4] linted deploymentMenu.ts --- src/deploymentMenu.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/deploymentMenu.ts b/src/deploymentMenu.ts index e10012d..100b7b0 100644 --- a/src/deploymentMenu.ts +++ b/src/deploymentMenu.ts @@ -495,7 +495,7 @@ async function deployIMCommand( ): Promise { const pipeAuth = `${obj.infName}-auth-pipe`; const imClientPath = await getIMClientPath(); - const templatePath = `$PWD/deployed-template.yaml`; + const templatePath = '$PWD/deployed-template.yaml'; let cmd = `%%bash PWD=$(pwd)