Test Package

Test package

The test package can be used to compare Vlasiator to reference runs, in order to see if results have changed. It does not look at physical quantities, it is purely a tool for tracking changes in results. Before a new version can be verified there has to be a set of verification data with which to compare.

1. Verify new version
2. Create new reference data
3. Porting to new machine

Compile vlasiator for testpackage

One should compile vlasiator with a conservative set of flags that make results more reproducible and also run the code in pure double precision mode. When verifying the code it is also important that the same set of flags has been used as for the verification data. On Voima and hornet one can compile it with the testpackage target to get the correct flags.

make clean
make testpackage -j 10

Verify Vlasiator

If everything is set up on the machine, one can copy the testpackage folder (with all of its contents) to a file-system where parallel jobs can be run, enter the folder and run the tests in two ways:

1. Interactively: ./small_test_machinename.sh
2. In queue system: qsub/sbatch small_test_machinename.sh

Creating new verification data

This can be done if a safe change is done which changes results, e.g., an algorithmic change.

1. Set in small_test_machinename.sh create_verification_files to 1
2. Run tests as usually
3. Update reference_revision in small_test_machninename.sh to the reference string given by the run, and set create_verification_files to 0.

Selecting which tests should be run (optional)

Editing the file small_test_definitions.sh you will find a line like run_tests=( 1 2 3 4 5 6 7 8 9 10 11 12 13 14 17 18 19) followed by settings being set for enumerated testpackage test simulations. You can edit the list contained within the run_tests variable to only run select tests, if necessary.

Porting testpackage to new machine

When moving to a new machine one has two options for generating reference data. Either one generates a new set of reference data which implies trust in the compiled version, or one can transfer old reference data from another system.

Adding testpackage target (optional)

You may also wish to generate a new set of conservative flags for the compiler. This can be done in the MAKE/Makefile.machinename file. For example: testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++0x -fabi-version=0

Porting testpackage driver script

For each new machine there needs to be a machine specific driver script. The existing ones are, e.g., small_test_voima.sh (Cray PBS), small_test_sisu.sh (Cray, slurm), and can be used as models for creating a new script.

1. Copy small_test_voima.sh to a new file small_test_machinename.sh.
2. In the beginning is PBS queue system commands, update these to another queuing system if needed. If run interactively these do not matter (like on laptop)
3. Update ht, t, and cores_per_node to reasonable values
4. Update run_command to match the system
5. Update the base reference_dir folder for storing reference results