Tests unable to distinguish build warnings, test warnings, and test pass/fail

[jarvisschultz]

System Info

• Operating System: Linux jschultz 4.15.0-66-generic #75~16.04.1-Ubuntu SMP Tue Oct 1 14:01:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
• Python Version: Python 2.7.12
• Version of catkin_tools: catkin_tools 0.4.5
• ROS Distro: kinetic

Build / Run Issue

• Works with catkin_make
• Works with catkin_make_isolated --merge
• Works with catkin build
• Works with catkin build -p1

Expected Behavior

Consider a CMake file with the following:

if(CATKIN_ENABLE_TESTING)
  catkin_add_gtest(test_${PROJECT_NAME}_utils test/test_source_code.cpp)
  add_dependencies(test_${PROJECT_NAME}_utils ${catkin_EXPORTED_TARGETS})
  target_link_libraries(test_${PROJECT_NAME}_utils ${PROJECT_NAME}_utils ${catkin_LIBRARIES})
endif()

The test_source_code.cpp implements several gtests with EXPECT_FALSE and EXPECT_TRUE tests. The utils library has logging capabilities that write to stderr when invalid data is passed to the methods. Ideally, catkin run_tests would be able to distinguish between passed/failed tests, compilation warnings, and logging statements. If a test is compiled with no warnings, and all tests pass then the summary and build output should properly reflect that.

Actual Behavior

All output from the verbose compilation and test running (see #405) is considered to be a warning. A --summarize incorrectly fails to report the tests for the package as Successful because of writing to stderr. You always end up with a [build] Warnings: 1 packages succeeded with warnings. at the end of compilation.

If libraries were abstracted differently, this problem could be avoided, but that isn't always desirable.