Fix Python linking on OSX

[Tobias-Fischer]

See e.g. ros-perception/vision_opencv#331 for reasoning

Fixes #805

[Tobias-Fischer]

/cc @raghavauppuluri13 @wolfv

[raghavauppuluri13]

I added the missing flags, but didn't know about removing the python linking. Nice!

[sloretz]

CI (repos: https://gist.githubusercontent.com/sloretz/5a5bb6c456cbbe4a1a13c9c48df25939/raw/9a8fe89ca80e3062354ee6f87211dfac8edd8dd4/ros2.repos build: --packages-up-to rclpy test: --packages-select rclpy)

• Linux
• Linux-aarch64
• macOS
• Windows

[sloretz]

LGTM with green CI

[Tobias-Fischer]

Note that we still face issues in ros2 doctor. Do you have any idea where else PYTHON_LIBRARIES are linked directly @sloretz by any chance?

[sloretz]

Do you have any idea where else PYTHON_LIBRARIES are linked directly @sloretz by any chance?

maybe here? https://github.com/ros2/rosidl_python/blob/17de859d09fdddc611f003757a9df1acab865cdf/rosidl_generator_py/cmake/rosidl_generator_py_generate_interfaces.cmake#L174

[sloretz]

@Tobias-Fischer Any idea why OSX CI for this PR had a segfault in one of the rclpy tests? I don't see the same issue on the nightly OSX job.

projectroot.test_c_handle

-- run_test.py: invoking following command in '/Users/osrf/jenkins-agent/workspace/ci_osx/ws/build/rclpy':
 - /Users/osrf/jenkins-agent/workspace/ci_osx/ws/build/rclpy/test_c_handle --gtest_output=xml:/Users/osrf/jenkins-agent/workspace/ci_osx/ws/build/rclpy/test_results/rclpy/test_c_handle.gtest.xml
dyld: Symbol not found: _PyExc_MemoryError
  Referenced from: /Users/osrf/jenkins-agent/workspace/ci_osx/ws/install/rclpy/lib/librclpy_common.dylib
  Expected in: flat namespace
 in /Users/osrf/jenkins-agent/workspace/ci_osx/ws/install/rclpy/lib/librclpy_common.dylib
-- run_test.py: return code -6

[Tobias-Fischer]

Unfortunately I have no idea re this exception.

Any clues @traversaro @wolfv?

[traversaro]

It may be worth to check if the Python executable used for the tests is the same Python whose libraries (and headers) have been used to compile the extension.

[traversaro]

It may be worth to check if the Python executable used for the tests is the same Python whose libraries (and headers) have been used to compile the extension.

The headers are particularly tricky, as even if CMake appears to have found the correct Python, the incorrect Python headers could be accidentally be used by compilation unit if the directory of the wrong Python is in the include path as it is installed in the prefix of some other dependency. This is however is just an hypothesis.

By inspecting the CI log, it seems that both Python 3.8 (the one that is then used to pip install dependencies and to create a venv) and Python 3.7 (that instead should be never used) are installed:

python 3.7.7
[email protected] 3.8.2

I am not sure if this is playing a role, perhaps it could make sense to try to uninstall the Python 3.7 if possible?

[traversaro]

However, I quickly checked and _PyExc_MemoryError is defined in any Python > 3.6 (see https://github.com/python/cpython/blob/3.6/Objects/exceptions.c#L2332), so it may be also worth to print the symbols exported (https://stackoverflow.com/questions/4506121/how-to-print-a-list-of-symbols-exported-from-a-dynamic-library) by the used Python library to understand if _PyExc_MemoryError is indeed provided or not.

[wolfv]

Maybe the difference is that we remove the "DYLD_LIBRARY_PATH" and in "vanilla ROS" it's set to some value? Just guessing