OCCT Automatic Testing System

OpenCASCADE released their automatic testing system both on their git repository and in official releases. But many input files come from their customers and are confidential, so they are not available.

Running the testsuite

OpenCASCADE automatic testing system is based on tcl scripts launched in DRAWEXE. Unlike OCE Testing Suite, there is no need to compile tests, so there is no specific configuration option to build tests. To run tests, check OCE_USE_TCL_TEST_FRAMEWORK from the cmake-gui frontend, then run:

$ make occt-tests

The OCE_USE_TCL_TEST_FRAMEWORK CMake option will generate two files in the build directory:

• DrawLaunchTests.draw contains DRAWEXE commands to run all tests. Normally it contains only the following two lines

testgrid
return

• DrawLaunchTests.cmake is a CMake script which runs

  DRAWEXE -f DrawLaunchTests.draw

  after setting several environment variables to tell DRAWEXE where to find libraries and resource files.

When

$ make occt-tests

is executed, it runs

cmake -P DrawLaunchTests.cmake

so one can also directly type the latter command.

Layout of tests

There are three levels of tests:

1. Test case: a single script to test one function (for instance: caf bugs A1)
2. Grid: a grid is a set of individual test cases, to test different inputs of a function (for instance: caf bugs)
3. Group: a group is a set of grids, to test related features (for instance: demo)

Analyzing results

Running testsuite will create in the build directory a directory named results_branch name_date, for instance results_master_2014-01-12T2321; the main report is stored in a file summary.html, which contains links to output logs and individual reports.

On Linux, output looks like:

	Status	Description
157	BAD	Known problem
2	FAILED	Test failed (regression)
1	IMPROVEMENT	Possible improvement (expected problem not detected)
4939	OK	Test passed OK
11476	SKIPPED	Test skipped due to lack of data file

Failed:
  bugs fclasses	bug7287_5
  demo samples

Improvements:
  boolean bopfuse_simple ZP6

In the report, there are links to detailed output, so that one can read the error message. Here, bug7287_5 complains about memory leaks, and samples about missing bottle.tcl sample file.

Debugging

By default, tests are run in parallel. One can specify the number of parallel tasks by adding a -parallel n option to testgrid in DrawLaunchTests.draw; for instance to run tests sequentially, replace testgrid by testgrid -parallel 1. This is the first thing to try in order to see if errors come from parallel execution. It is also possible to edit DrawLaunchTests.draw to run all tests of a given group:

testgrid bugs

or all tests of a single grid:

testgrid demo samples

In order to run a single test, the command testgrid must be replaced by test:

test bugs fclasses bug7287_5

But test result is then only printed to console, no log file is written in the results directory.