[Java] Dataset Failed to update reservation while freeing bytes: JNIEnv was not attached to current thread #20379
Comments
|
David Li / @lidavidm: For here: why do we need the NativeMemoryPool? IMO, now that we have C Data, we could just get rid of it. We just need to solve ARROW-16673. |
|
Error spotted when trying to use Arrow-dataset to add support of parquet file in Kotlin Dataframe : Kotlin/dataframe#577 (comment) |
|
Here is the C++-demangled version of the stack trace from the issue description: |
|
The problem is quite clear and should be relatively easy to fix. It comes from here: arrow/java/dataset/src/main/cpp/jni_wrapper.cc Lines 138 to 146 in 0dbbd43 See the JNI APIs: |
|
Thanks for the hint @pitrou , I'll try to add arrow::Status OnRelease(int64_t size) override {
JNIEnv* env;
if (vm_->GetEnv(reinterpret_cast<void**>(&env), JNI_VERSION) != JNI_OK) {
if (AttachCurrentThread(vm_,&env) != JNI_OK) {
return arrow::Status::Invalid("JNIEnv was not attached to current thread");
}
}
env->CallObjectMethod(java_reservation_listener_, unreserve_memory_method, size);
RETURN_NOT_OK(arrow::dataset::jni::CheckException(env));
return arrow::Status::OK();
} |
fb64
added a commit
to fb64/arrow
that referenced
this issue
Feb 16, 2024
…eing bytes: JNIEnv was not attached to current thread
fb64
added a commit
to fb64/arrow
that referenced
this issue
Feb 16, 2024
…eing bytes: JNIEnv was not attached to current thread
fb64
added a commit
to fb64/arrow
that referenced
this issue
Feb 16, 2024
…eing bytes: JNIEnv was not attached to current thread
fb64
added a commit
to fb64/arrow
that referenced
this issue
Feb 19, 2024
…eing bytes: JNIEnv was not attached to current thread
lidavidm
pushed a commit
that referenced
this issue
Feb 20, 2024
…ytes (#40101) ### Rationale for this change Better controls JNI Thread management in java dataset module to fix #20379 Re-use the same code found in the java arrow-c-data module : https://github.com/apache/arrow/blob/main/java/c/src/main/cpp/jni_wrapper.cc#L107 May JNIEnvGuard class code can be put in a common place for both libraries ... ### What changes are included in this PR? N/A ### Are these changes tested? These changes has been tested with : https://gist.github.com/fb64/71880cde297bc5234b02b68b785670fd on Linux X86_64 architecture ### Are there any user-facing changes? N/A * Closes: #20379 Authored-by: Florian Bernard <[email protected]> Signed-off-by: David Li <[email protected]>
zanmato1984
pushed a commit
to zanmato1984/arrow
that referenced
this issue
Feb 28, 2024
…eing bytes (apache#40101) ### Rationale for this change Better controls JNI Thread management in java dataset module to fix apache#20379 Re-use the same code found in the java arrow-c-data module : https://github.com/apache/arrow/blob/main/java/c/src/main/cpp/jni_wrapper.cc#L107 May JNIEnvGuard class code can be put in a common place for both libraries ... ### What changes are included in this PR? N/A ### Are these changes tested? These changes has been tested with : https://gist.github.com/fb64/71880cde297bc5234b02b68b785670fd on Linux X86_64 architecture ### Are there any user-facing changes? N/A * Closes: apache#20379 Authored-by: Florian Bernard <[email protected]> Signed-off-by: David Li <[email protected]>
thisisnic
pushed a commit
to thisisnic/arrow
that referenced
this issue
Mar 8, 2024
…eing bytes (apache#40101) ### Rationale for this change Better controls JNI Thread management in java dataset module to fix apache#20379 Re-use the same code found in the java arrow-c-data module : https://github.com/apache/arrow/blob/main/java/c/src/main/cpp/jni_wrapper.cc#L107 May JNIEnvGuard class code can be put in a common place for both libraries ... ### What changes are included in this PR? N/A ### Are these changes tested? These changes has been tested with : https://gist.github.com/fb64/71880cde297bc5234b02b68b785670fd on Linux X86_64 architecture ### Are there any user-facing changes? N/A * Closes: apache#20379 Authored-by: Florian Bernard <[email protected]> Signed-off-by: David Li <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Using Dataset NativeMemoryPool.getDefault() work very well with any size of data input as you see in:
In case we decided to use NativeMemoryPool.createListenable we are seeing this error message: /Users/runner/work/crossbow/crossbow/arrow/java/dataset/src/main/cpp/jni_util.cc:78: Failed to update reservation while freeing bytes: JNIEnv was not attached to current thread:
Log stack trace:
Reporter: David Dali Susanibar Arce / @davisusanibar
Note: This issue was originally created as ARROW-17508. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: