Add new pass/fail critera for test cases for TRIBITS_ADD_ADVANCED_TEST() #150
Assignees
Labels
Comments
|
Wow, this is crazy but I found out that the I am going to fix this as part of this story as well. |
bartlettroscoe
added a commit
that referenced
this issue
Oct 23, 2016
These tests are not in Teuchos and have not been for many years.
bartlettroscoe
added a commit
that referenced
this issue
Oct 23, 2016
This commit adds and modifies the behavior of TRIBITS_ADD_ADVANCED_TEST() (TAAT()). * Added TAAT() TEST_<IDX> pass/fail option WILL_FAIL that has the same behaviorx as the built-in CTest WILL_FAIL option. * Made TAAT() TEST_<IDX> argument FAIL_REGULAR_EXPRESSION work. Before, that argument was completely ignored. I also made this behavior the way that the raw CTest FAIL_REGULAR_EXPRESSION argument work. * Added some tests for TRIBITS_ADD_TEST() that checked and verified the behavior of CTest for the different pass/fail options. I needed these tests to figure out how raw CMake/CTest behaves with respect to the test properties PASS_REGULAR_EXPRESSION, FAIL_REGULAR_EXPRESSION, and WILL_FAIL. * Due to defect #82, I set DISABLE_CIRCULAR_REF_DETECTION_FAILURE for the TriBITS package so that I could test the behavior of FAIL_REGULAR_EXPRESSION. I will fix that defect next. * Make every TAAT() pass/fail criteria print [PASSED] or [FAILED].
bartlettroscoe
added a commit
that referenced
this issue
Oct 23, 2016
* Added the argument ALWAYS_FAIL_ON_NONZERO_RETURN (see #150) * Fixed documentation for behavior of WILL_FAIL and FAIL_REGULAR_EXPRESSION (see previous commit). * Added section to explain pass/fail logic for a test case. * Renamed TEST_PASSED to TEST_CASE_PASSED to make more clear. * Fixed a few mispelled words in documentation (after running spell-check).
|
I should have this reviewed but I don't know who would be willing to review it. |
bartlettroscoe
added a commit
that referenced
this issue
Oct 29, 2016
It was doing the correct test case pass/fail but was not printing the right pass/fail for this check. Now it does. I did not add a unit test for this since I think this is a more minor issue (and I am not sure how to test it). Build/Test Cases Summary Enabled Packages: Enabled all Packages 0) MPI_DEBUG => passed: passed=215,notpassed=0 (0.47 min) 1) SERIAL_RELEASE => passed: passed=215,notpassed=0 (0.43 min)
|
No one is going to review this. Closing as complete. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
While implementing the tests for RPATH handling (#126), I identified that we really new a few new pass/fail criteria for TRIBITS_ADD_ADVANCED_TEST() for each individual
TEST_<IDX>case:ALWAYS_FAIL_ON_NONZERO_RETURN: Require a zero return value in addition to other pass/fail criteria. (Often times you want to grep for output but you would also like the return value to be zero in addition. As of CMake 3.7, CTest tests don’t have a pass/fail property like this so this can only be added the TRIBITS_ADD_ADVANCED_TEST().)WILL_FAIL: Fail the test if it returns a zero error code. (Same behavior as built-in propertyWILL_FAIL).These test cases will be helpful for other tests as well.
The text was updated successfully, but these errors were encountered: