diff --git a/README.md b/README.md index 8de5a5f..b21a277 100644 --- a/README.md +++ b/README.md @@ -12,11 +12,47 @@ Automated virtual experiment that calculates geometric properties of nanoporous ## Launching the virtual experiment -First you need to import the virtual experiment in your ST4SD registry [from the global ST4SD registry](https://st4sd.github.io/overview/using-the-virtual-experiments-registry-ui). You will then be able to start the parameterised package `nanopore-geometry-experiment` in your private ST4SD registry (see [example notebook](nanopore-geometry-experiment.ipynb)). +### On a local machine + +## Instructions + +If you have a container runtime such as docker, podman, rancher desktop, etc available on your system you can execute the experiment by: + +1. creating a python virtual environment, activating it, and installing the python module `st4sd-runtime-core[develop]` +2. cloning this repository +3. launching the experiment + +For example: + +```bash +#!/usr/bin/env sh + +: # Download virtual experiment +git clone https://github.com/st4sd/nanopore-geometry-experiment.git + +: # Setup ST4SD runtime-core +python3 -m venv --copies venv +. venv/bin/activate +python3 -m pip install "st4sd-runtime-core[develop]" + +: # Run experiment +elaunch.py -i docker-example/cif_files.dat \ + --applicationDependencySource="nanopore-database=cif:copy" \ + nanopore-geometry-experiment.package +``` + +**Note**: Make sure you run the `git clone` command in a directory that your container runtime (e.g.`docker`, `podman`, etc) can mount later when you execute the experiment. + +**Note**: Using a container runtime is intended for small scale experiments and/or debugging your experiments. Use the `kubernetes` example below to launch larger experiments. + +### On Kubernetes + +Follow our end to end [example notebook](nanopore-geometry-experiment.ipynb) to launch our `nanopore-geometry-experiment` experiment on your ST4SD Cloud instance. + ## Using custom database of CIF files -You may download the CIF files of your choosing to a PVC inside your OpenShift cluster (below we use the name `nanopore-database-pvc`), mount it as a volume and ask the virtual experiment instance to use the contents of the PVC as the contents of the `nanopore-database` application-dependency. You will also need to remove the line related to `nanopore-database` from the `manifest.yaml` to enable the use of the PVC. +You may download the CIF files of your choosing to a PVC inside your OpenShift cluster (below we use the name `nanopore-database-pvc`), mount it as a volume and ask the virtual experiment instance to use the contents of the PVC as the contents of the `nanopore-database` application-dependency. ```Python file_names = [""] @@ -32,8 +68,6 @@ rest_uid = api.api_experiment_start("nanopore-geometry-experiment", payload) **Note**: The [example notebook](nanopore-geometry-experiment.ipynb) shows a full example. -**Note**: You can also run a small scale experiment using docker/podman. See our example [here](docker-example/README.md). - ## Help and Support Please feel free to create an issue and alert the maintainers listed in the [MAINTAINERS.md](MAINTAINERS.md) page. diff --git a/cif/README.md b/cif/README.md index e57a756..f73a57a 100644 --- a/cif/README.md +++ b/cif/README.md @@ -2,7 +2,7 @@ ## Bundled CIF files -It is possible to run the experiments by adding the required CIF files to this folder and using it as `nanopore-database` dependency in the `manifest.yaml`. +It is possible to run the experiments by adding the required CIF files to this folder. Then, you can define the materials that are relevant to the experiment setting the content of `cif_files.dat` in the `Define a subset of materials to be screened` section of the [notebook](nanopore-geometry-experiment.ipynb). diff --git a/docker-example/README.md b/docker-example/README.md deleted file mode 100644 index b9a8ca6..0000000 --- a/docker-example/README.md +++ /dev/null @@ -1,36 +0,0 @@ -# Run nanopore-geometry-experiment using the Docker backend of ST4SD - -## Prerequisites - -1. A recent version of python 3 - [python 3.7+](https://www.python.org/downloads/) -2. The [docker](https://docs.docker.com/get-docker/) container runtime -3. The [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) command-line utility - - -## Instructions - -You can try out the experiment on your laptop by: - -1. creating a python virtual environment, activating it, and installing the python module `st4sd-runtime-core` -2. cloning this repository -3. launching the experiment - - -For example: - -```bash -#!/usr/bin/env sh - -# Download virtual experiment -git clone https://github.com/st4sd/nanopore-geometry-experiment.git - -# Setup ST4SD runtime-core -python3 -m venv --copies venv -. venv/bin/activate -python3 -m pip install "st4sd-runtime-core" - -# Run experiment -./nanopore-geometry-experiment/docker-example/run.sh -``` - -**Note**: Make sure you run the `git clone` command in a directory that `docker` (or `podman`) can mount later when you execute the experiment. \ No newline at end of file diff --git a/docker-example/run.sh b/docker-example/run.sh deleted file mode 100755 index 67a7724..0000000 --- a/docker-example/run.sh +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/env sh -cd $(dirname ${0})/.. - -docker_like_location="" -if ! which docker >/dev/null; then - echo "The docker command is not available on this system, will try to use podman" >&2 - - if location=$(which podman); then - docker_like_location="--dockerExecutableOverride=${location}" - echo "Podman located at ${location}" >&2 - else - echo "No docker or podman command on this system - aborting" >&2 - exit 2 - fi -fi - -# Run experiment -echo "Executing the experiment ..." >&2 -elaunch.py --failSafeDelays=no ${docker_like_location} -l40 \ - --instanceName nanopore-geometry-experiment \ - --manifest manifest.yaml \ - --input docker-example/cif_files.dat \ - --platform docker conf/flowir_package.yaml - -# See outputs of experiment -echo "\n\nMeasured properties are:" >&2 -output_dir=$(ls -td nanopore-geometry-experiment*.instance | head -1) -cat "${output_dir}/output/properties.csv" diff --git a/manifest.yaml b/manifest.yaml deleted file mode 100644 index 9dffdf2..0000000 --- a/manifest.yaml +++ /dev/null @@ -1,2 +0,0 @@ -nanopore-database: ../cif:copy -hooks: ../hooks:copy diff --git a/nanopore-geometry-experiment.ipynb b/nanopore-geometry-experiment.ipynb index 03a591f..5881752 100644 --- a/nanopore-geometry-experiment.ipynb +++ b/nanopore-geometry-experiment.ipynb @@ -150,20 +150,19 @@ " \"git\": {\n", " \"location\": {\n", " \"url\": \"https://github.com/st4sd/nanopore-geometry-experiment.git\",\n", - " \"commit\": \"21410c4b1fbfc8e7645984df97f27431dcad88bb\"\n", + " \"tag\": \"1.1.0\"\n", " }\n", " }\n", " },\n", " \"config\": {\n", - " \"path\": \"conf/flowir_package.yaml\",\n", - " \"manifestPath\": \"manifest.yaml\"\n", + " \"path\": \"nanopore-geometry-experiment.package\"\n", " }\n", " }]\n", " },\n", " \"metadata\": {\n", " \"package\": {\n", " \"name\": \"nanopore-geometry-experiment\",\n", - " \"tags\": [\"latest\", \"1.0.0\"],\n", + " \"tags\": [\"latest\", \"1.1.0\"],\n", " \"maintainer\": \"rneumann@br.ibm.com\",\n", " \"license\": \"Apache 2.0\",\n", " \"description\": \"Uses the Zeo++ software package to calculate geometric properties of nanoporous materials.\",\n", @@ -212,14 +211,17 @@ "\n", "experimentConfiguration = {\n", "# Uncomment the \"volumes\" field below if you want to use a PVC instead of the bundled cif files.\n", - "# It will also be necessary to comment the \"nanopore-database: ../cif:copy\" line in manifest.yaml\n", + "# The PVC should contain your `CoRE2019` directory\n", "# \"volumes\": [{\n", "# \"type\": {\"persistentVolumeClaim\": \"nanopore-database-pvc\"},\n", "# \"applicationDependency\": \"nanopore-database\"\n", "# }],\n", " \"additionalOptions\": [\n", " \"--useMemoization=true\",\n", - " \"--registerWorkflow=true\"\n", + " \"--registerWorkflow=true\",\n", + "\n", + " # Comment out this line if you've uncommented the \"volumes\" lines above\n", + " \"--applicationDependencySource=nanopore-database=%(package)s/../cif:copy\"\n", " ],\n", " \"orchestrator_resources\": {\n", " \"cpu\": \"1.0\",\n", @@ -303,7 +305,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.8" + "version": "3.10.13" }, "nteract": { "version": "0.28.0" diff --git a/conf/flowir_package.yaml b/nanopore-geometry-experiment.package/conf/flowir_package.yaml similarity index 98% rename from conf/flowir_package.yaml rename to nanopore-geometry-experiment.package/conf/flowir_package.yaml index 8e32d79..a660846 100644 --- a/conf/flowir_package.yaml +++ b/nanopore-geometry-experiment.package/conf/flowir_package.yaml @@ -95,12 +95,12 @@ interface: variables: default: global: - numberOfNanopores: 3 probeRadius_A: 1.4 zeo_memory: '2Gi' - docker: - global: numberOfNanopores: 1 + openshift: + global: + numberOfNanopores: 3 blueprint: openshift: @@ -109,7 +109,7 @@ blueprint: config: walltime: 14400.0 backend: kubernetes - docker: + default: global: resourceManager: config: @@ -266,6 +266,7 @@ output: platforms: - openshift - - docker + - docker # inherits everything from "default" platform + - default version: 1.0.0 diff --git a/hooks/interface.py b/nanopore-geometry-experiment.package/hooks/interface.py similarity index 100% rename from hooks/interface.py rename to nanopore-geometry-experiment.package/hooks/interface.py diff --git a/bin/aggregate_geometry_output.py b/reference-scripts/aggregate_geometry_output.py similarity index 100% rename from bin/aggregate_geometry_output.py rename to reference-scripts/aggregate_geometry_output.py diff --git a/bin/geo_amalgamate.py b/reference-scripts/geo_amalgamate.py similarity index 100% rename from bin/geo_amalgamate.py rename to reference-scripts/geo_amalgamate.py diff --git a/bin/modules/parse_zeopp.py b/reference-scripts/modules/parse_zeopp.py similarity index 100% rename from bin/modules/parse_zeopp.py rename to reference-scripts/modules/parse_zeopp.py diff --git a/bin/write_geometry_to_database.py b/reference-scripts/write_geometry_to_database.py similarity index 100% rename from bin/write_geometry_to_database.py rename to reference-scripts/write_geometry_to_database.py diff --git a/bin/zeo.sh b/reference-scripts/zeo.sh similarity index 100% rename from bin/zeo.sh rename to reference-scripts/zeo.sh