Skip to content

Test Management

Jump to bottom
David Jansen edited this page Nov 5, 2019 · 7 revisions

Introduction / Overview

These slides give an excellent overview about possible solutions to test Modelica libraries in a good manner.

Unit Tests

We want to make sure that all models in the library work correctly and that changes do not introduce unwanted side effects to exiting models or break them. Instead of testing each model manually, we use the unit testing functions of the Python package BuildingsPy to automate the process. This process is very important for quality control of our library development.

A unit test for a Modelica model in our case consists of an example model that is automatically simulated by a *.mos-Script and a Python routine that compares the simulation results of the example model with stored reference results. Thus, if something causes the example model's results to change, this can be automatically detected.

Setting up your system for running the unit tests

This section assumes you are on a Windows system and have Dymola and Python installed. As BuildingsPy currently only supports Python 2, the Python installation should be a recent version of Python 2.

In order to run the unit tests of AixLib you will need a few things:

Have BuildingsPy installed

When installing BuildingsPy (and ModelicaRes), we recommend to clone their respective repositories. You should then have a local version of them at <path-to-your-cloned-repository>.Then install the packages by typing pip install -e <path-to-your-cloned-repository> into your cmd prompt. This is because other AixLib developers will also work with the current master of these packages from the repositories instead of with current official release versions. Therefore, using pip install buildingspy or pip install modelicares may lead to you using different versions than other AixLib developers.

Important: Currently for the coverage determination we have a line of code in our runUnitTests.py which is not supported by the original Buildingspy library. Please comment out this line for running unittests.

Have pytidylib installed and libtidy.dll set up

pytidylib should automatically be collected with the installation of the BuildingsPy package. Otherwise, you can install it with pip install pytidylib. Unfortunately, on Windows you also have to make sure to set up libtidy.dll. To do this, follow these steps:

Have matplotlib and numpy installed

As fundamental changes to these packages happen more rarely, it is ok to install them by simply typing pip install matplotlib and pip install numpy into your cmd prompt. That is just in case you do not have them already installed, which is the default for many Python distributions. In that case, it does not hurt to update to the latest release with pip install numpy --upgrade or pip install matplotlib --upgrade .

Have your PATH environmental variable point to your Dymola executable

In order to be able to start Dymola for the simulation of the unit test example model, PATH must point to the Dymola executable. Therefore, add e.g. C:\Program Files (x86)\Dymola 2015 FD01\bin64 to your PATH variable.

With all this in place, you should be ready to go and be able to run the unit tests.

Run the unit tests

In order to run the unit tests open a command prompt and cd to the directory into which you cloned the AixLib repository. Let's call this directory <path-to-AixLib>. In <path-to-AixLib> there should be the directory AixLib that contains the package.mo file for the library. Next, cd to AixLib, so that your command prompt is now at <path-to-AixLib>\AixLib. From here, you can run the unit tests by typing Resources\Scripts\runUnitTests.py.

As a default, the unit tests simulate the example models and compare the simulation results with the reference results. In addition, the unit tests check the model documentations for correct html syntax. In some cases, it may be useful not to run all functions of the unit tests, but only part of it. For this, the unit test script offers a few modifiers:

Run unit tests to validate html syntax only, without simulating models

When updating the documentation of models, it may be faster to only run the html syntax check on the models, without simulating all the models. To do this, run the unit tests with the modifier --validate-html-only. Thus, from <path-to-AixLib>\AixLib execute ..\bin\runUnitTests.py --validate-html-only.

Run unit tests in batch mode, without user interaction

In normal operation, the unit tests will require a user interaction if any changes from the reference results are detected. To avoid this user interaction, use the modifier -b or --batch. Thus, from <path-to-AixLib>\AixLib execute ..\bin\runUnitTests.py -b.

Specify the number of processors available for the unit tests

As default, the unit tests will use all available processors. If you want to limit the number of processors used for the unit tests, use the modifier -n <number-of-processors>. Thus, for example for using only 2 processors, from <path-to-AixLib>\AixLib execute ..\bin\runUnitTests.py -n 2.

Run unit tests for a subpackage only, not for all of AixLib

Especially for development, it may be useful to only run the unit tests for a specific subpackage, instead of for the whole library. To do this, use the modifier -s <AixLib.package>, with <AixLib.package> being a model path in dot notation. E.g. to only test the low order building models, from <path-to-AixLib>\AixLib execute ..\bin\runUnitTests.py -s AixLib.Building.LowOrder. This will test the specified package and all its subpackages.

New Results

You should accept new results for your new unit test and commit these results before submitting a pull request. If results have changed for existing unit tests, only accept the changes if you can explain them and make sure to document the reasons for the change on the issue tracker/ in the pull request.

Table of contents

Clone this wiki locally