AAR packaged libc++_shared.so?

[henryzx]

Issue description

The libc++_shared.so shipped with fbjni.aar conflicts with our app's.
According to android official doc, they recommended libraries to ship native code as a single so file.
Want to know why we don't static link it to single fbjni.so?
And is there any suggested way to resolve native conflicts like this?

thanks.

System Info

Android

[bogerchan]

+1

[ussernamenikita]

+1

[passy]

I've played around with -DANDROID_STL=c++_static and that's not compatible with our other projects, like Flipper which builds further shared libraries based on it.

I'm seeing errors like this:

prefabandroid.armeabi-v7a: Library is a shared library with a statically linked STL and cannot be used with any library using the STL

Unless we find a solution for that, we won't be able to change the default for our distribution.

[cpsauer]

Related to #62

[cortinico]

Piggybacking here as we're facing the same exact issue on React Native. We're looking into consume fbjni as a Prefab:

• Consume Fbjni via Prefab facebook/react-native#33397

I think the best approach here would be to ship 2 distributions of fbjni, one with libc++_shared.so and one without (that's what Hermes is doing). As an alternative, just remove libc++_shared.so from fbjni entirely as suggested by the Android documentation:
https://developer.android.com/ndk/guides/cpp-support#shared_runtimes

Caution: JNI libraries distributed with Java AARs must not use the shared runtime to avoid conflicting with other libraries and the app. The warnings below still apply. See the documentation for Middleware Vendors for more information.

We're currently blocked on React Native with a:

> A failure occurred while executing com.android.build.gradle.internal.tasks.MergeNativeLibsTask$MergeNativeLibsTaskWorkAction
   > 2 files found with path 'lib/arm64-v8a/libc++_shared.so' from inputs:
      - /root/.gradle/caches/transforms-3/94fdb9ae737c1ffb0e2b5a7d92619e93/transformed/jetified-react-native-1000.0.0-d1bc3bfd2/jni/arm64-v8a/libc++_shared.so
      - /root/.gradle/caches/transforms-3/e97ae185b4d1b38cfd833e7ebc155a69/transformed/jetified-fbjni-0.2.2/jni/arm64-v8a/libc++_shared.so
     If you are using jniLibs and CMake IMPORTED targets, see
     https://developer.android.com/r/tools/jniLibs-vs-imported-targets

ref: https://app.circleci.com/pipelines/github/facebook/react-native/12400/workflows/eb260b13-6ae7-4927-a915-4be5c0643027/jobs/239232

And we have to resort to the current non-prefab approach, which relies on unbundling the libraries manually (the one described in this repo docs) which is essentially equivalent to drop the libc++_shared.so library provided in the artifact.

[cpsauer]

@cortinico I think they mean something slightly different by that quote, especially given the linked middleware guide. (For all others, it's super worth a read! It succinctly answers the "whys" above. And it seems like we might be conflating bundling the shared C++ standard library with linking against it.)

To have fbjni not link, use, and need the shared C++ standard library, we'd want to add an option to statically link fbjni into apps or libraries that use it. That could be done by distributing static archives (or just compiling the source) and giving the option to set the shared library name on the Java side (as mentioned in #62 (comment)). We might also want to provide a linker script to help people preserve the Java->C++ JNI entry points. (See https://developer.android.com/ndk/guides/middleware-vendors#using_the_stl). It seems like the Android guides would recommend this for most use cases, but fbjni isn't currently distributed this way, unfortunately.

[Otherwise, fbjni has to dynamically link against the C++ standard library, since the standard library is (unfortunately) part of its interface, as per the middleware guide. (Another (probably worse) fix would be moving to a no-STL interface, as the guide notes.) Hopefully that answers @henryzx's initial "why" question, above.]

All that said, it seems like maybe the app and using-library cases could be solved by just dynamically linking against (the same) C++ standard library as fbjni, bundled or not? The real pain comes when using two libraries that want different standard libraries.

[cortinico]

Following up here @cpsauer

All that said, it seems like maybe the app and using-library cases could be solved by just dynamically linking against (the same) C++ standard library as fbjni, bundled or not?

According to this answer android/ndk#1283 (comment) from the folks on the NDK, the recommended approach is to use libc++_shared.so (as FBJNI is doing right now). So yeah, the current bundling seems to be the preferred one.

The libc++_shared.so shipped with fbjni.aar conflicts with our app's.

To solve @henryzx problem, he could technically use a pickFirst{} block and let AGP decide which libc++_shared.so to pick. The problem is that by doing so you don't allow a developer to pick the libc++ from the library they wish (as AGP will decide for them).

AGP will prioritize libc++_shared.so from the app, if it exists (it should print a warning). But if you import two libraries (say React Native and FBJNI) which both imports libc++_shared.so then you're forced to use a pickFirst{}.

The solution I was suggesting was to publish 2 version/flavors of FBJNI (one with the bundled libc++_shared.so and one without) to overcome this.

[cpsauer]

👍🏻 I suppose more generally, is it clear that FBJNI should be bundling libc++_shared at all, as opposed to just requiring it?

[cortinico]

👍🏻 I suppose more generally, is it clear that FBJNI should be bundling libc++_shared at all, as opposed to just requiring it?

could you clarify? As I'm not sure if you missed a not in your sentence.

To reiterate: I believe FBJNI could bundle libc++_shared.so, but should also offer a way to be consumed without.

[atsushieno]

Back in June I created this library and published some AARs that only contain libc++_shared.so. Does this look helpful? https://github.com/atsushieno/libcxx-provider/

[cortinico]

Back in June I created this library and published some AARs that only contain libc++_shared.so. Does this look helpful? https://github.com/atsushieno/libcxx-provider/

Sadly not, as we need to don't have libc++_shared.so bundled at all in any dependency modulo the App (or React Native). Adding libcxx provider will add yet another libc++_shared.so

[atsushieno]

What I really meant is that libcxx-provider would eliminate the last issue that prevents migration to "Prefab packaging without libcxx_shared.so" model. (Thus my comment primarily targets FBJNI developers, not its consumers.)

libcxx-provider is not suitable if the app project builds its own C++ code, given that the dependencies do not bundle libc++_shared.so. You're right on that it will only result in bringing extraneous libc++_shared.so on the table.
It is useful when no deps bundle libc++_shared.so AND there is no externalNativeBuild in the app either (which then results in UnsatisfiedLinkError due to missing libc++_shared.so). For such a case libcxx-provider is useful.

To my understanding the UnsatisfiedLinkError is the only problematic case that prevents Prefab and "do not bundle STL" approach. Old prefabs did not mix with non-prefab AARs, but now they do, if I understand correctly.