Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Application crashes at the end of execution when using GPU (OpenVINO 2020.4) #675

Open
cavusmustafa opened this issue Oct 7, 2020 · 1 comment

Comments

@cavusmustafa
Copy link

cavusmustafa commented Oct 7, 2020

ltrace_out.tar.gz
ngraph_log.tar.gz
strace_out.tar.gz

When the backend device set to GPU, the application crashed just before the execution finishes. The crash happens in the destructor of "IE_Backend" when calling "m_exec_map.clear();" (line 44 of ie_backend.cc).

The issue is observed when using a GPU only. CPU and HDDL runs without any error.

The crash happens at the end of the execution. I was able to execute the inference and see the results.

The backend device is set by the line below in the python script:
ngraph_bridge.set_backend(<device_name>)

The error message:
terminate called without an active exception
./run_inception_v3.sh: line 7: 7113 Aborted (core dumped) python -u infer_frozen.py -m inception_v3.pb -i input:0 -o InceptionV3/Predictions/Softmax:0 --images images.txt --num-iterations 10 --inf-time -d $device

Models used to observe this issue:
resnet_v2_50
inception_v3

The commit id of the branch I was using: 15a32ba
OpenVINO version: 2020.4

Edit: Uploaded the outputs of strace, ltrace, and ngraph bridge logs.

@cavusmustafa
Copy link
Author

The issue might be related to "m_exec_map" in IE_Backend. The pointers to the executables are already caches in "m_ng_exec_map" in NGraphEncapsulateImpl. Simply not adding any entry to "m_exec_map" prevents this error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant