refactor: make package fit "standalone" format and add instructions t…

…o run locally (#2) * feat: update package to fit "standard" project format Signed-off-by: Vassilis Vassiliadis <[email protected]> * chore: update the commit in the notebook Signed-off-by: Vassilis Vassiliadis <[email protected]> * fix: fix syntax in notebook Signed-off-by: Vassilis Vassiliadis <[email protected]> * fix: use a tag Signed-off-by: Vassilis Vassiliadis <[email protected]> * chore: move bin dir outside of .package dir as these scripts are not used Signed-off-by: Vassilis Vassiliadis <[email protected]> --------- Signed-off-by: Vassilis Vassiliadis <[email protected]>
st4sd · Jun 20, 2024 · 95d1518 · 95d1518
1 parent 85244f5
commit 95d1518
Show file tree

Hide file tree

Showing 13 changed files with 54 additions and 83 deletions.
diff --git a/README.md b/README.md
@@ -12,11 +12,47 @@ Automated virtual experiment that calculates geometric properties of nanoporous
 
 ## Launching the virtual experiment
 
-First you need to import the virtual experiment in your ST4SD registry [from the global ST4SD registry](https://st4sd.github.io/overview/using-the-virtual-experiments-registry-ui). You will then be able to start the parameterised package `nanopore-geometry-experiment` in your private ST4SD registry (see [example notebook](nanopore-geometry-experiment.ipynb)).
+### On a local machine
+
+## Instructions
+
+If you have a container runtime such as docker, podman, rancher desktop, etc available on your system you can execute the experiment by:
+
+1. creating a python virtual environment, activating it, and installing the python module `st4sd-runtime-core[develop]`
+2. cloning this repository
+3. launching the experiment
+
+For example:
+
+```bash
+#!/usr/bin/env sh
+
+: # Download virtual experiment
+git clone https://github.com/st4sd/nanopore-geometry-experiment.git
+
+: # Setup ST4SD runtime-core
+python3 -m venv --copies venv
+. venv/bin/activate
+python3 -m pip install "st4sd-runtime-core[develop]"
+
+: # Run experiment
+elaunch.py -i docker-example/cif_files.dat \
+      --applicationDependencySource="nanopore-database=cif:copy" \
+      nanopore-geometry-experiment.package
+```
+
+**Note**: Make sure you run the `git clone` command in a directory that your container runtime (e.g.`docker`, `podman`, etc) can mount later when you execute the experiment.
+
+**Note**: Using a container runtime is intended for small scale experiments and/or debugging your experiments. Use the `kubernetes` example below to launch larger experiments.
+
+### On Kubernetes
+
+Follow our end to end [example notebook](nanopore-geometry-experiment.ipynb) to launch our `nanopore-geometry-experiment` experiment on your ST4SD Cloud instance.
+
 
 ## Using custom database of CIF files
 
-You may download the CIF files of your choosing to a PVC inside your OpenShift cluster (below we use the name `nanopore-database-pvc`), mount it as a volume and ask the virtual experiment instance to use the contents of the PVC as the contents of the `nanopore-database` application-dependency. You will also need to remove the line related to `nanopore-database` from the `manifest.yaml` to enable the use of the PVC.
+You may download the CIF files of your choosing to a PVC inside your OpenShift cluster (below we use the name `nanopore-database-pvc`), mount it as a volume and ask the virtual experiment instance to use the contents of the PVC as the contents of the `nanopore-database` application-dependency.
 
 ```Python
 file_names = [""]
@@ -32,8 +68,6 @@ rest_uid = api.api_experiment_start("nanopore-geometry-experiment", payload)
 
 **Note**: The [example notebook](nanopore-geometry-experiment.ipynb) shows a full example.
 
-**Note**: You can also run a small scale experiment using docker/podman. See our example [here](docker-example/README.md).
-
 ## Help and Support
 
 Please feel free to create an issue and alert the maintainers listed in the [MAINTAINERS.md](MAINTAINERS.md) page.

diff --git a/cif/README.md b/cif/README.md
@@ -2,7 +2,7 @@
 
 ## Bundled CIF files
 
-It is possible to run the experiments by adding the required CIF files to this folder and using it as `nanopore-database` dependency in the `manifest.yaml`.
+It is possible to run the experiments by adding the required CIF files to this folder.
 
 Then, you can define the materials that are relevant to the experiment setting the content of `cif_files.dat` in the `Define a subset of materials to be screened` section of the [notebook](nanopore-geometry-experiment.ipynb).
 

diff --git a/docker-example/README.md b/docker-example/README.md
diff --git a/docker-example/run.sh b/docker-example/run.sh
diff --git a/manifest.yaml b/manifest.yaml
diff --git a/nanopore-geometry-experiment.ipynb b/nanopore-geometry-experiment.ipynb
@@ -150,20 +150,19 @@
     "        \"git\": {\n",
     "          \"location\": {\n",
     "            \"url\": \"https://github.com/st4sd/nanopore-geometry-experiment.git\",\n",
-    "            \"commit\": \"21410c4b1fbfc8e7645984df97f27431dcad88bb\"\n",
+    "            \"tag\": \"1.1.0\"\n",
     "          }\n",
     "        }\n",
     "      },\n",
     "      \"config\": {\n",
-    "        \"path\": \"conf/flowir_package.yaml\",\n",
-    "        \"manifestPath\": \"manifest.yaml\"\n",
+    "        \"path\": \"nanopore-geometry-experiment.package\"\n",
     "      }\n",
     "    }]\n",
     "  },\n",
     "  \"metadata\": {\n",
     "    \"package\": {\n",
     "      \"name\": \"nanopore-geometry-experiment\",\n",
-    "      \"tags\": [\"latest\", \"1.0.0\"],\n",
+    "      \"tags\": [\"latest\", \"1.1.0\"],\n",
     "      \"maintainer\": \"[email protected]\",\n",
     "      \"license\": \"Apache 2.0\",\n",
     "      \"description\": \"Uses the Zeo++ software package to calculate geometric properties of nanoporous materials.\",\n",
@@ -212,14 +211,17 @@
     "\n",
     "experimentConfiguration = {\n",
     "# Uncomment the \"volumes\" field below if you want to use a PVC instead of the bundled cif files.\n",
-    "# It will also be necessary to comment the \"nanopore-database: ../cif:copy\" line in manifest.yaml\n",
+    "# The PVC should contain your `CoRE2019` directory\n",
     "#     \"volumes\": [{\n",
     "#         \"type\": {\"persistentVolumeClaim\": \"nanopore-database-pvc\"},\n",
     "#         \"applicationDependency\": \"nanopore-database\"\n",
     "#     }],\n",
     "    \"additionalOptions\": [\n",
     "        \"--useMemoization=true\",\n",
-    "        \"--registerWorkflow=true\"\n",
+    "        \"--registerWorkflow=true\",\n",
+    "\n",
+    "        # Comment out this line if you've uncommented the \"volumes\" lines above\n",
+    "        \"--applicationDependencySource=nanopore-database=%(package)s/../cif:copy\"\n",
     "    ],\n",
     "    \"orchestrator_resources\": {\n",
     "      \"cpu\": \"1.0\",\n",
@@ -303,7 +305,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.8"
+   "version": "3.10.13"
   },
   "nteract": {
    "version": "0.28.0"

diff --git a/conf/flowir_package.yaml → ...periment.package/conf/flowir_package.yaml b/conf/flowir_package.yaml → ...periment.package/conf/flowir_package.yaml
@@ -95,12 +95,12 @@ interface:
 variables:
   default:
     global:
-      numberOfNanopores: 3
       probeRadius_A: 1.4
       zeo_memory: '2Gi'
-  docker:
-    global:
       numberOfNanopores: 1
+  openshift:
+    global:
+      numberOfNanopores: 3
 
 blueprint:
   openshift:
@@ -109,7 +109,7 @@ blueprint:
         config:
           walltime: 14400.0
           backend: kubernetes
-  docker:
+  default:
     global:
       resourceManager:
         config:
@@ -266,6 +266,7 @@ output:
 
 platforms:
   - openshift
-  - docker
+  - docker # inherits everything from "default" platform
+  - default
 
 version: 1.0.0
diff --git a/hooks/interface.py → ...try-experiment.package/hooks/interface.py b/hooks/interface.py → ...try-experiment.package/hooks/interface.py
diff --git a/bin/aggregate_geometry_output.py → ...ence-scripts/aggregate_geometry_output.py b/bin/aggregate_geometry_output.py → ...ence-scripts/aggregate_geometry_output.py
diff --git a/bin/geo_amalgamate.py → reference-scripts/geo_amalgamate.py b/bin/geo_amalgamate.py → reference-scripts/geo_amalgamate.py
diff --git a/bin/modules/parse_zeopp.py → reference-scripts/modules/parse_zeopp.py b/bin/modules/parse_zeopp.py → reference-scripts/modules/parse_zeopp.py
diff --git a/bin/write_geometry_to_database.py → ...nce-scripts/write_geometry_to_database.py b/bin/write_geometry_to_database.py → ...nce-scripts/write_geometry_to_database.py
diff --git a/bin/zeo.sh → reference-scripts/zeo.sh b/bin/zeo.sh → reference-scripts/zeo.sh