Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libloading support #63

Open
chertov opened this issue Feb 26, 2021 · 1 comment
Open

libloading support #63

chertov opened this issue Feb 26, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@chertov
Copy link

chertov commented Feb 26, 2021

Hi! I'm trying to use bindgen dynamic libloading for onnx shared library runtime.
I want to be able load and unload onnx shared library at runtime.
my current work is here chertov@b94870f

Can we make onnx shared library features check before loading?
For example we can check the existence of symbols like this

OrtSessionOptionsAppendExecutionProvider_CoreML
OrtSessionOptionsAppendExecutionProvider_CUDA
OrtSessionOptionsAppendExecutionProvider_Nnapi
etc..

in the ONNX runtime shared library and wrap this information with methods like "is_cuda_support()"

@chertov
Copy link
Author

chertov commented Feb 26, 2021

Simple CoreML demo in this branch
https://github.com/chertov/onnxruntime-rs/tree/libloading
i removed global variables g_ort(), G_ENV
running Session with CoreML https://github.com/chertov/onnxruntime-rs/blob/libloading/onnxruntime/src/session.rs#L117
but it falls with error:

OrtSessionOptionsAppendExecutionProvider_CoreML is ok = true
OrtSessionOptionsAppendExecutionProvider_CUDA is ok = false
OrtSessionOptionsAppendExecutionProvider_Nnapi is ok = true
Feb 26 19:14:29.957 DEBUG new{api=OrtApi name="test" log_level=Info}: onnxruntime::environment: Environment created. env_ptr="0x7feda0f84500"
Feb 26 19:14:29.958 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="226" function="operator()" logid=""}: onnxruntime::onnxruntime: "Flush-to-zero and denormal-as-zero are off"
Feb 26 19:14:29.958 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="233" function="ConstructorCommon" logid=""}: onnxruntime::onnxruntime: "Creating and using per session threadpools since use_per_session_threads_ is true"
Feb 26 19:14:29.966 DEBUG onnxruntime{category="onnxruntime" file="bfc_arena.cc" line_number="23" function="BFCArena" logid="test"}: onnxruntime::onnxruntime: "Creating BFCArena for CoreML with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 memory limit: 18446744073709551615 arena_extend_strategy 0"
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="bfc_arena.cc" line_number="23" function="BFCArena" logid="test"}: onnxruntime::onnxruntime: "Creating BFCArena for CoreML with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 memory limit: 18446744073709551615 arena_extend_strategy 0"
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="1107" function="Initialize" logid=""}: onnxruntime::onnxruntime: "Initializing session."
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="inference_session.cc" line_number="1132" function="Initialize" logid=""}: onnxruntime::onnxruntime: "Adding default CPU execution provider."
Feb 26 19:14:29.967 DEBUG onnxruntime{category="onnxruntime" file="bfc_arena.cc" line_number="23" function="BFCArena" logid="test"}: onnxruntime::onnxruntime: "Creating BFCArena for Cpu with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 memory limit: 18446744073709551615 arena_extend_strategy 0"
Feb 26 19:14:29.970 DEBUG onnxruntime{category="onnxruntime" file="reshape_fusion.cc" line_number="37" function="ApplyImpl" logid=""}: onnxruntime::onnxruntime: "Total fused reshape node count: 0"
Feb 26 19:14:29.971 DEBUG onnxruntime{category="onnxruntime" file="reshape_fusion.cc" line_number="37" function="ApplyImpl" logid=""}: onnxruntime::onnxruntime: "Total fused reshape node count: 0"
Feb 26 19:14:29.972 DEBUG onnxruntime{category="onnxruntime" file="coreml_execution_provider.cc" line_number="166" function="GetCapability" logid=""}: onnxruntime::onnxruntime: "CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 10 number of nodes in the graph: 65 number of nodes supported by CoreML: 53"
Feb 26 19:14:29.987 DEBUG drop{self=SessionBuilder { env: Environment { env: Mutex { data: EnvironmentSingleton { name: "test", env_ptr: 0x7feda0f84500, api: OrtApi } } }, session_options_ptr: 0x7feda2904460, allocator: Arena, memory_type: Default }}: onnxruntime::session: Dropping the session options.
Error: Failed to create session: Error calling ONNX Runtime C function: Error compiling model compiler error:  Invalid network: Layer name missing.

@nbigaouette nbigaouette added the enhancement New feature or request label Mar 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants