DUNE Run Control User Interface (drunc-ui)

This repo defines the web interface for various drunc tools. Including:

• The Processes Manager
• The Controller

Docker Setup

This repository provides a Docker Compose configuration providing the requisite services suitable for development of the interfaces using realistic test data. Working with the full docker stack requires building a very larger docker base image and has heavy resource requirements when running. For this reason a "lite" stack is also provided that can be used for development of the Process Manager UI.

The full and lite stack are defined using Docker Compose profiles. It is recommended to set the COMPOSE_PROFILES environment variable to select which stack you are working with. Use COMPOSE_PROFILES=drunc for the full stack or COMPOSE_PROFILES=drunc-lite for the lite stack.

You can also use the --profile flag when invoking docker compose however the instructions in this file assume you are using the environment variable.

Process Manager UI

You can use either the full or lite Docker Compose profiles when working with the Process Manager UI.

Getting Started with the Process Manager UI

Check you have a value set for COMPOSE_PROFILES (see above). Start the docker stack with:

docker compose up

It can take a few moments for the services to boot but the application should then be available in the browser at http://localhost:8000.

Note that if you have the Process Manager UI already open in a browser you may see backtraces from unsuccessful requests generated by the UI. These should stop once drunc has finished starting up.

Authentication is required to work with the application so you need to create a user account to work with:

docker compose exec app python manage.py createsuperuser

and follow the prompts. You should then be able to use the details you supplied to pass the login screen. You can use the "boot" button on the main page to create simple processes to experiment with.

Take the services down with docker compose down or by pressing Ctrl+C in the corresponding terminal.

Controller UI

You must use the full Docker Compose profile when working with the Controller UI.

Getting Started with the Controller UI

Check you have COMPOSE_PROFILES=drunc (see above). As with the Process Manager UI start the Docker Compose services and create a user account if not already done. You can then boot a full test session:

docker compose exec drunc /boot_test_session.sh

Note that the above consumes several Gb of memory.

Once booted you can interact with the root controller via:

docker compose exec drunc /entrypoint.sh drunc-controller-shell grpc://localhost:3333

For details of working with the controller see the drunc wiki. You should also be able to see the booted processes in the Process Manager UI.

Development

Working with the full functionality of the web application requires a number of services to be started and to work in concert. The Docker Compose stack provides the required services and is suitable for development and manual testing but is not suitable for running QA (pre-commit) tooling or unit tests. The project directory is mounted into the app service which allows the Django development server's auto-reload mechanism to detect changes to local files and work as expected.

It is recommended that you follow the below instructions on working with poetry to run the project's QA tooling and Unit Tests.

Working with Poetry

This is a Python application that uses poetry for packaging and dependency management. It also provides pre-commit hooks for various linters and formatters and automated tests using pytest and GitHub Actions. Pre-commit hooks are automatically kept updated with a dedicated GitHub Action.

To get started:

1. Download and install Poetry following the instructions for your OS.

2. Clone this repository and make it your working directory

3. Set up the virtual environment:

   poetry install

4. Activate the virtual environment (alternatively, ensure any Python-related command is preceded by poetry run):

   poetry shell

5. Install the git hooks:

   pre-commit install

Pre-commit should now work as expected when making commits even without the need to have an active poetry shell. You can also manually run pre-commit (e.g. pre-commit run -a) and the unit tests with pytest. Remember you'll need to prefix these with poetry run first if you don't have an active poetry shell.

Running the web application with Poetry

You can also start the web application though at a minimum this requires the drunc process manager to be running. Note that drunc only works on Linux so this approach will not work on any other platforms. See the next section on also working with Kafka. Assuming you have an active poetry shell for all steps:

1. Start the drunc shell:

   drunc-unified-shell --log-level debug ./data/process-manager-no-kafka.json

2. In another terminal, run the main app:

   python manage.py runserver

3. As above you'll need to create a user to get past the login page:

   python manage.py createsuperuser

Note that if you boot any processes in the web application this will immediately die with an exit code of 255. This is because the drunc shell requires an ssh server on localhost in order to be able to run processes. In most cases this isn't very limiting.

Running the web application with Poetry and Kafka

In the event that you want to work with the full application without using Docker Compose you must start the required components manually. Assuming you have an active poetry shell for all steps.

1. Start Kafka - See Running drunc with pocket kafka.

2. Start the drunc shell: drunc-unified-shell --log-level debug ./data/process-manager-pocket-kafka.json

3. Start the application server: python manage.py runserver

4. Start the Kafka consumer: python manage.py kafka_consumer --debug

From here you should be able to see broadcast messages displayed at the top of the index page on every refresh.