Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use exist_ok=True in makedirs instead of checking for existence #212

Conversation

emersonknapp
Copy link
Contributor

This was a tricky one to track down, as it happened extremely rarely.

The case is, you've got two different Python parameter modules you're generating within a single CMake project

generate_parameter_module(first_generated_params
  ${PROJECT_NAME}/first_params.yaml
)
generate_parameter_module(second_generated_params
  ${PROJECT_NAME}/second_params.yaml
)

When make finally is autorun by colcon build, it's done with parallelization and the two commands can actually be run simultaneously. When this happened (and it was rare but not as uncommon as I would have thought, definitely not reliably reproducible but would fail CI runs sporadically) it looks like this:

Running generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py install/my_package/lib/python3.8/site-packages/my_package/first_generated_params.py src/my_pkg/my_pkg/first_params.yaml
Running generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py install/my_pkg/lib/python3.8/site-packages/my_pkg/second_generated_params.py src/my_pkg/my_pkg/second_params.yaml
Traceback (most recent call last):
  File "generate_parameter_library_py/bin/generate_parameter_library_python", line 33, in <module>
    sys.exit(load_entry_point('generate-parameter-library-py', 'console_scripts', 'generate_parameter_library_python')())
  File "generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py", line 66, in main
    run(output_file, yaml_file, validate_file)
  File "generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py", line 44, in run
    os.makedirs(output_dir)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: 'install/my_pkg/lib/python3.8/site-packages/my_pkg'
make[2]: *** [CMakeFiles/second_generated_parameters.dir/build.make:71: install/my_pkg/lib/python3.8/site-packages/my_pkg/second_generated_parameters.py] Error 1

This fix mitigates that concern by using a non-throwing call to the makedirs API

…, in case of multiple concurrent processes

Signed-off-by: Emerson Knapp <[email protected]>
@pac48 pac48 merged commit bfc22d5 into PickNikRobotics:main Jul 24, 2024
5 of 7 checks passed
@jcarpinelli-bdai
Copy link

Thank you! Do you plan on backporting this to humble?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants