Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Mobile] CreatMemoryInfo Failure in Android when try to create Ort::Value #21097

Closed
Windsander opened this issue Jun 19, 2024 · 6 comments
Closed

[Mobile] CreatMemoryInfo Failure in Android when try to create Ort::Value #21097

Windsander opened this issue Jun 19, 2024 · 6 comments
Labels
platform:mobile issues related to ONNX Runtime mobile; typically submitted using template

Comments

@Windsander
Copy link

Windsander commented Jun 19, 2024 •
edited
Loading

Describe the issue

When running the ONNXRuntime on an Android device, a segmentation fault occurs with the following stack trace:

art_sigsegv_fault 0x00000076c5912654
art::FaultManager::HandleFault(int, siginfo *, void *) 0x00000076c5912b74
art::SignalChain::Handler(int, siginfo *, void *) 0x0000007766e48328
__kernel_rt_sigreturn 0x000000776985e888
Ort::MemoryInfo::CreateCpu(OrtAllocatorType, OrtMemType) onnxruntime_cxx_inline.h:303
onnx::sd::base::TensorHelper::create<…>(std::__ndk1::vector<…>, std::__ndk1::vector<…>) onnxsd_basic_tools.cc:213
onnx::sd::context::OrtSD_Context::OrtSD_Remain::OrtSD_Remain() ort_sd_context.cc:43
onnx::sd::context::OrtSD_Context::OrtSD_Context(const onnx::sd::context::OrtSD_Config &) ort_sd_context.cc:73
generate_context ort_sd.cc:16
inferenceSD(int, const char **) native-lib.cpp:620
Java_com_arikanli_project_1orion_NativeInferenceEngine_inferenceSD(JNIEnv *, jobject, jobjectArray) native-lib.cpp:704
art_quick_generic_jni_trampoline 0x00000076c5822248
art_quick_invoke_stub 0x00000076c5818968
art::ArtMethod::Invoke(art::Thread *, unsigned int *, unsigned int, art::JValue *, const char *) 0x00000076c58845ac
art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread *, art::ArtMethod *, art::ShadowFrame *, unsigned short, art::JValue *) 0x00000076c59fd6a0
art::interpreter::DoCall<…>(art::ArtMethod *, art::Thread *, art::ShadowFrame &, const art::Instruction *, unsigned short, art::JValue *) 0x00000076c59f8514
art::interpreter::ExecuteSwitchImplCpp<…>(art::interpreter::SwitchImplContext *) 0x00000076c582c640
ExecuteSwitchImplAsm 0x00000076c5824bdc
art::interpreter::ExecuteSwitch(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool) 0x00000076c59f7ef4
art::interpreter::Execute(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool, bool) 0x00000076c59f00a4
art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame *, art::JValue *) 0x00000076c59f7a84
art::interpreter::DoCall<…>(art::ArtMethod *, art::Thread *, art::ShadowFrame &, const art::Instruction *, unsigned short, art::JValue *) 0x00000076c59f84f0
art::interpreter::ExecuteSwitchImplCpp<…>(art::interpreter::SwitchImplContext *) 0x00000076c582d7ac
ExecuteSwitchImplAsm 0x00000076c5824bdc
art::interpreter::ExecuteSwitch(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool) 0x00000076c59f7ef4
art::interpreter::Execute(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool, bool) 0x00000076c59f00a4
art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame *, art::JValue *) 0x00000076c59f7a84
art::interpreter::DoCall<…>(art::ArtMethod *, art::Thread *, art::ShadowFrame &, const art::Instruction *, unsigned short, art::JValue *) 0x00000076c59f84f0
art::interpreter::ExecuteSwitchImplCpp<…>(art::interpreter::SwitchImplContext *) 0x00000076c582dc0c
ExecuteSwitchImplAsm 0x00000076c5824bdc
art::interpreter::ExecuteSwitch(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool) 0x00000076c59f7ef4
art::interpreter::Execute(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool, bool) 0x00000076c59f00a4
art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame *, art::JValue *) 0x00000076c59f7a84
art::interpreter::DoCall<…>(art::ArtMethod *, art::Thread *, art::ShadowFrame &, const art::Instruction *, unsigned short, art::JValue *) 0x00000076c59f84f0
art::interpreter::ExecuteSwitchImplCpp<…>(art::interpreter::SwitchImplContext *) 0x00000076c582dc0c
ExecuteSwitchImplAsm 0x00000076c5824bdc
art::interpreter::ExecuteSwitch(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool) 0x00000076c59f7ef4
art::interpreter::Execute(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool, bool) 0x00000076c59f00a4
art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame *, art::JValue *) 0x00000076c59f7a84
art::interpreter::DoCall<…>(art::ArtMethod *, art::Thread *, art::ShadowFrame &, const art::Instruction *, unsigned short, art::JValue *) 0x00000076c59f84f0
art::interpreter::ExecuteSwitchImplCpp<…>(art::interpreter::SwitchImplContext *) 0x00000076c582d7ac
ExecuteSwitchImplAsm 0x00000076c5824bdc
art::interpreter::ExecuteSwitch(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool) 0x00000076c59f7ef4
art::interpreter::Execute(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool, bool) 0x00000076c59f00a4
art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame *, art::JValue *) 0x00000076c59f7a84
art::interpreter::DoCall<…>(art::ArtMethod *, art::Thread *, art::ShadowFrame &, const art::Instruction *, unsigned short, art::JValue *) 0x00000076c59f84f0
art::interpreter::ExecuteSwitchImplCpp<…>(art::interpreter::SwitchImplContext *) 0x00000076c582d7ac
ExecuteSwitchImplAsm 0x00000076c5824bdc
art::interpreter::ExecuteSwitch(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool) 0x00000076c59f7ef4
art::interpreter::Execute(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool, bool) 0x00000076c59f00a4
art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame *, art::JValue *) 0x00000076c59f7a84
art::interpreter::DoCall<…>(art::ArtMethod *, art::Thread *, art::ShadowFrame &, const art::Instruction *, unsigned short, art::JValue *) 0x00000076c59f84f0
art::interpreter::ExecuteSwitchImplCpp<…>(art::interpreter::SwitchImplContext *) 0x00000076c582d7ac
ExecuteSwitchImplAsm 0x00000076c5824bdc
art::interpreter::ExecuteSwitch(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool) 0x00000076c59f7ef4
art::interpreter::Execute(art::Thread *, const art::CodeItemDataAccessor &, art::ShadowFrame &, art::JValue, bool, bool) 0x00000076c59f00a4
art::interpreter::

seems it caused by Ort::MemoryInfo::CreateCpu(OrtAllocatorType, OrtMemType) method fail.

I'm using libonnxruntime.so directly in native-cpp android lib, by link with ort-sd.a, which created from below project:
https://github.com/Windsander/CFDI-StableDiffusionONNXFast.git
and using cross compile in Android-native project with cmakelists.txt like this:

cmake_minimum_required(VERSION 3.22.1)
project("project_orion")

add_subdirectory(${CMAKE_SOURCE_DIR}/ort-sd)

add_library(${CMAKE_PROJECT_NAME} SHARED
        # List C/C++ source files with relative paths to this CMakeLists.txt.
        native-lib.cpp
)

link_directories(${CMAKE_SOURCE_DIR}/libs)

# Specifies libraries CMake should link to your target library. You
# can link libraries from various origins, such as libraries defined in this
# build script, prebuilt third-party libraries, or Android system libraries.
target_link_libraries(${CMAKE_PROJECT_NAME}
        # List libraries link to the target library
        android
        log
        ort-sd
)

really needs some help to resolve this issue 0x0

the libonnxruntime.so comes from official 1.18.0-mobile:
https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-android
in .arr/jni dir

To reproduce

https://github.com/Windsander/CFDI-StableDiffusionONNXFast.git

Urgency

No response

Platform

Android

OS Version

1.18.0

ONNX Runtime Installation

Released Package

Compiler Version (if 'Built from Source')

No response

Package Name (if 'Released Package')

None

ONNX Runtime Version or Commit ID

1.18.0-release

ONNX Runtime API

C++/C

Architecture

ARM64

Execution Provider

Default CPU

Execution Provider Library Version

No response
@Windsander Windsander added the platform:mobile issues related to ONNX Runtime mobile; typically submitted using template label Jun 19, 2024
@Windsander
Copy link
Author

Any help would be greatly appreciated. )x0

@Windsander
Copy link
Author

I've already tried manually initialize ortAPI, by:

#define ORT_API_MANUAL_INIT
......
OrtSD_Context::OrtSD_Context(const OrtSD_Config& ort_config_){
#ifdef ORT_API_MANUAL_INIT
    Ort::InitApi(OrtGetApiBase()->GetApi(ORT_API_VERSION));
#endif
    ......
    Tensor empty_test = TensorHelper::create(TensorShape{0}, std::vector<float>{});
}

but seems not the reason for current trouble...

also failed at TensorHelper::create, which is an encapsulation method for:

        Tensor result_tensor_ = Tensor::CreateTensor<T>(
            Ort::MemoryInfo::CreateCpu(
                OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault
            ), result_data_, input_size_,
            shape_.data(), shape_.size()
        );

same error message occur at :

Ort::MemoryInfo::CreateCpu(OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault)

@Windsander
Copy link
Author

the ort-sd submodule CMakeLists.txt at here:
https://github.com/Windsander/CFDI-StableDiffusionONNXFast/blob/main/CMakeLists.txt

@Windsander
Copy link
Author

I finally identified the cause of the error by examining the suspicious function mappings between the library .o files and the corresponding header .h files. The issue was that I included the onnxruntime/include files directly, which turned out to be incorrect:
image

THIS IS NOT THE TURE HEAD FILES, CAUSE THE TRUELY ONE IS DYNAMIC GENERATED FROM LIB CMAKE PROCESSING, AND SOME NEEDS MACRO DEFINES FROM CMAKE PROCESS

The correct header files are dynamically generated during the library's CMake processing, and some of them require macro definitions from the CMake process. Therefore, the necessary includes should be obtained from the appropriate build directory.

If you download ONNX Runtime from a release(like me), make sure to use the headers generated during the build process, not the ones directly from the onnxruntime/include directory.

the necessary include should be here, if download onnxruntime from release is here:
image

This was referenced Jun 21, 2024
Open
Closed
@skottmckay
Copy link
Contributor

The ORT headers aren't dynamically generated AFAIK, but you should always use the headers from the package with the ORT library to ensure they match.

@Windsander
Copy link
Author

The ORT headers aren't dynamically generated AFAIK, but you should always use the headers from the package with the ORT library to ensure they match.

If I want to generate the onnxruntime library locally, where can I get the headers?
I can confirm that the issue is caused by header mismatch. Now, I have resolved the function matching for the online-release download using the aforementioned method.
But how can I solve this for local generation?
Thanks. 0x0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
platform:mobile issues related to ONNX Runtime mobile; typically submitted using template
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@skottmckay @Windsander