47: Add README.md and ci scripts for system_tests and example_sdk_cli…

…ent providing easy info and access to using them locally. Also add Legal considerations in root README.md and alter reconnect.py example to fully show reconnect.

README.md

@@ -63,3 +63,19 @@ Job cc17549b-2d11-4076-904e-bd548b456e57 is done (type: grow_optimizer). Status:
 2024-02-13 17:27:19,004 [aio_pika.queue][Thread-1][queue.py:526][INFO]: <RobustQueueIterator: queue='jobs.cc17549b-2d11-4076-904e-bd548b456e57.status' ctag='ctag1.66d19448a3354b1292fec482738b9e23'> closing with timeout 5 seconds
 2024-02-13 17:27:19,011 [omotes_sdk][Thread-1][broker_interface.py:228][INFO]: Stopped broker interface
 ```
+
+# Licensing & legal considerations
+We have opensourced the OMOTES stack under GPLv3. This ensures that the components of OMOTES are not
+altered and run closed-source but changes to OMOTES components are required to also be opensourced.
+However, the GPLv3 license does not prevent from OMOTES being used in a closed-source, commercial
+setting. The OMOTES stack is orchestrated by using one of the OMOTES SDK packages to communicate
+over the network with OMOTES. As such, the copy-left virality of the GPLv3 license does not apply
+to any application or system which integrates and uses OMOTES through the SDK packages. Each of the
+SDK packages is licensed using a permissive license. This is all done with the goal of ensuring an
+opensource, common calculation and model backend which is available to all but allow companies to
+use OMOTES without legal hurdles.
+
+In case you have any bug fixes or generally-usable extensions, it would be greatly appreciated
+if you make these changes available to one of the relevant OMOTES repositories so we can integrate
+them for all. The licenses, however, do not require changes to be opensourced to the original
+OMOTES repositories and it is up to the developer on how to opensource any changes.

example_sdk_client/README.md

@@ -0,0 +1,17 @@
+# Example clients using the SDK
+This directory contains a selection of scripts that use the Python SDK with a specific
+functionality. It is meant to provide an easy starting point on how to integrate the Python
+SDK (or other SDKs) in your own application.
+
+In order to run the examples, it is necessary to setup the Python SDK first. You may
+run `./ci/linux/create_venv.sh` to create a virtual environment or just simply
+`pip install -r requirements.txt`. Then (activate the virtual environment if necessary and)
+run any of the example scripts through python such as `python3 optimizer.py`.
+
+The scripts show case the following functionalities:
+- `optimizer.py`: Start an optimization job and wait for the result.
+- `optimizer_cancel.py`: Start an optimization job and cancel it after some time.
+- `reconnect.py`: Reconnect the SDK after prior submitting a job and stopping the SDK. This
+  simulates the case where an application (which integrates the SDK) has to reboot for some reason.
+  It then needs to reconnect to any submitted but unfinished jobs to receive the events.
+- `simulator.py`: Start a simulation job and wait for the result.

example_sdk_client/ci/linux/create_venv.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env sh
+
+python3 -m venv ./.venv
+if [[ "$OSTYPE" != "win32" && "$OSTYPE" != "msys" ]]; then
+  echo "Activating .venv first."
+  . .venv/bin/activate
+fi
+pip3 install -r ./requirements.txt

example_sdk_client/reconnect.py

@@ -43,20 +43,51 @@ def handle_on_progress_update(job: Job, progress_update: JobProgressUpdate):
     )
 
 
+workflow_optimizer = WorkflowType("grow_optimizer_default", "some descr")
+workflow_manager = WorkflowTypeManager([workflow_optimizer])
+
+
 try:
-    workflow_simulator = WorkflowType("simulator", "some descr")
-    workflow_manager = WorkflowTypeManager([workflow_simulator])
+    omotes_if_1 = OmotesInterface(rabbitmq_config, possible_workflows=workflow_manager)
+    with open("example_esdl_optimizer_poc_tutorial.esdl") as open_file:
+        input_esdl = open_file.read()
+
+    omotes_if_1.start()
+    submitted_job: Job = omotes_if_1.submit_job(
+        esdl=input_esdl,
+        params_dict={"key1": "value1", "key2": ["just", "a", "list", "with", "an", "integer", 3]},
+        workflow_type=workflow_optimizer,
+        job_timeout=None,
+        callback_on_finished=handle_on_finished,
+        callback_on_progress_update=handle_on_progress_update,
+        callback_on_status_update=handle_on_status_update,
+        auto_disconnect_on_result=True,
+    )
+    time.sleep(1)  # Wait for the message to be send.
+finally:
+    omotes_if_1.stop()
 
-    omotes_if = OmotesInterface(rabbitmq_config, possible_workflows=workflow_manager)
-    omotes_if.start()
+print(
+    f"Disconnected the Omotes interface from SDK after submitting job {submitted_job.id}. "
+    f"Will reconnect soon."
+)
+print(
+    "Any events we are currently missing are queued in RabbitMQ and will be waiting for us when "
+    "we reconnect."
+)
+time.sleep(5)
+print("Reconnecting...")
 
-    omotes_if.connect_to_submitted_job(
-        job=Job(id=UUID("81b1fec2-60b0-468a-804f-fc06c533fbb2"), workflow_type=workflow_simulator),
+try:
+    omotes_if_2 = OmotesInterface(rabbitmq_config, possible_workflows=workflow_manager)
+    omotes_if_2.start()
+    omotes_if_2.connect_to_submitted_job(
+        job=Job(id=submitted_job.id, workflow_type=workflow_optimizer),
         callback_on_finished=handle_on_finished,
         callback_on_progress_update=handle_on_progress_update,
         callback_on_status_update=handle_on_status_update,
         auto_disconnect_on_result=True,
     )
     time.sleep(60)
 finally:
-    omotes_if.stop()
+    omotes_if_2.stop()

system_tests/README.md

@@ -0,0 +1,15 @@
+# OMOTES System Tests
+
+This directory contains a number of tests which test the OMOTES stack/system as a whole. It
+requires a running OMOTES stack and it interacts with OMOTES through the python SDK as if it is
+any other (frontend) application.
+
+# Running the system tests
+In order to run the system tests, first ensure there is an OMOTES stack running. RabbitMQ
+should be exposed on localhost:5672 (which it is by default). It also requires the default
+login credentials omotes:somepass1 to log into RabbitMQ as an SDK.
+
+```bash
+./ci/linux/create_venv.sh # Sets up the dependencies for running the system tests application.
+./ci/linux/run.sh         # Runs the system tests. May be repeated.
+```

system_tests/ci/linux/create_venv.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env sh
+
+python3 -m venv ./.venv
+if [[ "$OSTYPE" != "win32" && "$OSTYPE" != "msys" ]]; then
+  echo "Activating .venv first."
+  . .venv/bin/activate
+fi
+pip3 install -r ./requirements.txt

system_tests/ci/linux/run.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+if [[ "$OSTYPE" != "win32" && "$OSTYPE" != "msys" ]]; then
+  echo "Activating .venv first."
+  . .venv/bin/activate
+fi
+cd src/
+pytest --timeout 120

system_tests/requirements.txt

@@ -2,3 +2,4 @@ omotes_sdk_python==0.0.16
 pytest ~= 8.1.2
 pytest-timeout
 xmltodict ~= 0.13.0
+pip-tools