Pywr-next

This is repository contains the current work-in-progress for the next major revision to Pywr. It uses Rust as a backend instead of Cython. It is currently not ready for use beyond development and experimentation. Comments and discussions are welcome.

User Guide · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgments

About The Project

Pywr-1.x is a Python library which utilises Cython for performance. Over time this has resulted in a "core" set of data structures and objects that are written in Cython to gain maximum performance. Cython has the nice benefit of making it easy to extend that core functionality using regular Python. However, the border between what is Python and what is Cython is a bit blurred and not well designed in certain places.

One option for the future development of Pywr (e.g. Pywr-2.x) would be a more explicit separation between the compute "core" and higher level functionality. Rust is a candidate for writing that core largely independent of Python, and possibly offers the benefits of (1) greater performance than Cython, and (2) easier maintenance in the future.

(back to top)

Requirements

Any major revision to Pywr will have the following feature requirements:

• Retain the "Parameter" system from Pywr-1.x - this is core functionality that makes Pywr really flexible.
• Extendable in Python space.
• An improved approach for outputting data and metrics.
• Better error handling.
• Cross-platform.
• Faster!
• Strong input file (JSON) schema.

Built With

(back to top)

Getting started

This repository contains a version of Clp using Git submodules. In order to build those submodules must be initialised first.

git submodule init
git submodule update

Rust is required for installation of the Python extension. To create a Python development installation requires first compiling the Rust library and then the Python extension. The following example uses a virtual environment to install the Python dependencies, compile the Pywr extension and run the Pywr Python CLI.

python -m venv .venv # create a new virtual environment
source .venv/bin/activate # activate the virtual environment (linux)
# .venv\Scripts\activate # activate the virtual environment (windows)
pip install maturin  # install maturin for building the Python extension
maturin develop # compile the Pywr Python extension
python -m pywr  # run the Pywr Python CLI

(back to top)

Usage

Rust CLI

A basic command line interface is included such that you can use this version of Pywr without Python. This CLI is in the pywr-cli crate.

To see the CLI commands available run the following:

cargo run -p pywr-cli -- --help

To run a Pywr v2 model use the following:

cargo run -p pywr-cli -- run tests/models/simple1.json

Python CLI

If the Python extension has been compiled using the above instructions a model can be run using the basic Python CLI.

python -m pywr run tests/models/simple1.json

Porting a Pywr v1.x model to v2.x

This version of Pywr is not backward compatible with Pywr v1.x. One of the major reasons for this version is the lack of a strong schema in the Pywr v1.x JSON files. Pywr v2.x uses an updated JSON schema that is defined in this repository. Therefore, v1.x JSON files must be converted to the v2.x JSON schema. This conversion can be undertaken manually, but there is also a work-in-progress conversion tool. The conversion tool uses a v1.x schema defined in the pywr-schema project.

Please note that conversion from Pywr v1.x to v2.x is experimental and not all features of Pywr are implemented in pywr-schema or have been implemented in Pywr v2.x yet. Due to the changes between these versions it is very likely an automatic conversion will not completely convert your model, and it WILL require manual testing and checking.

cargo run --no-default-features -- convert /path/to/my/v1.x/model.json

Feedback on porting models is very welcome, so please open an issue with any questions or problems.

(back to top)

Crates

This repository contains the following crates:

Pywr-core

A low-level Rust library for constructing network models. This crate interfaces with linear program solvers.

Feature flags:

Feature	Description	Default
highs	Enable the HiGHS LP solver.	False
ipm-ocl	Enable the OpenCL IPM solver (requires nightly).	False
ipm-simd	Enable the AVX IPM solver (requires nightly).	False

Pywr-schema

A Rust library for validating Pywr JSON files against a schema, and then building a model from the schema using pywr-core.

Feature flags:

Feature	Description	Default
core	Enable building models from the schema with pywr-core. This feature is enabled by default, but requires a lot of dependencies. If you only require schema validation and manipulation consider building this crate with default-features = false	True

Pywr-cli

A command line interface for running Pywr models.

Pywr-python

A Python extension (and package) for constructing and running Pywr models.

Roadmap

• Proof-of-concept - demonstrate the benefits of the RIIR approach.
• Redesign of outputs & metrics.
• Redesign of variable API for integration with external optimisation algorithms.
• Implement outstanding Parameters from Pywr v1.x
• Design & implement Python API using Rust extension.
• Release Pywr v2.x beta

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the Apache 2.0 or MIT License. See LICENSE.txt for more information.

(back to top)

Contact

James Tomlinson - [email protected]

Project Link: https://github.com/pywr/pywr-next

(back to top)

Copyright (C) 2020-2023 James Tomlinson Associates Ltd.