Version 1.3.268 does not load in app for which 1.3.236 loads correctly (Android 12)

[Izhido]

Environment:

• OS: Android 12 (Vulkan 1.1.128)
• GPU and driver version: Adreno 740, Driver 512.746.0

Describe the Issue

Loading version 1.3.268 of the validation layer lib causes an java.lang.UnsatisfiedLinkError at runtime. Latest known version that works, with unchanged settings, is 1.3.236.

Expected behavior

Valid Usage ID

Additional context

The reported error is as follows:

E  FATAL EXCEPTION: main
Process: com.my_namespace.my_app, PID: 16821
java.lang.UnsatisfiedLinkError: Unable to load native library "/data/app/~~eBT-ClU8d8rElTi09Pz4Rw==/com.my_namespace.my_app-9oOtV6cikWELznRSV6vYYA==/base.apk!/lib/arm64-v8a/libmy_app_native.so": dlopen failed: library "D:/Projects/MyApp/app/src/main/cpp/../../../../android-binaries-vulkan-sdk-1.3.268.0/arm64-v8a/libVkLayer_khronos_validation.so" not found: needed by /data/app/~~eBT-ClU8d8rElTi09Pz4Rw==/com.my_namespace.my_app-9oOtV6cikWELznRSV6vYYA==/base.apk!/lib/arm64-v8a/libmy_app_native.so in namespace classloader-namespace
	at android.app.NativeActivity.onCreate(NativeActivity.java:178)
	at android.app.Activity.performCreate(Activity.java:8057)
	at android.app.Activity.performCreate(Activity.java:8037)
	at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1341)
	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3705)
	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3881)
	at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:103)
	at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
	at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2270)
	at android.os.Handler.dispatchMessage(Handler.java:106)
	at android.os.Looper.loopOnce(Looper.java:214)
	at android.os.Looper.loop(Looper.java:304)
	at android.app.ActivityThread.main(ActivityThread.java:7918)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1010)

[Izhido]

UPDATE:
Thanks to the guys at the Vulkan Discord, I was able to build the (at the time) latest version of the library in my machine (1.3.275). Unfortunately, the new lib exhibits the same behavior as 1.3.268. I don't think I'm knowledgeable enough to find out what could be going on in the sources, so I most likely will need your help on finding out what's wrong.

[spencer-lunarg]

my only guess is this is related to the problem in #7340

I guess we slowly built up to many objects in the symbol namespace, this has been fixed on ToT main and best guess would be to try that version (since these changes didn't make it in the 1.3.275 SDK unfortunately)

[spencer-lunarg]

@Izhido looking into this more, it seems the reason it is failing is because the path name might be wrong. I can exhibit th same behavior if I change the name of the binary on my local Android test app

[Izhido]

@spencer-lunarg sorry for the way, way, waaay late answer. Now that we know that the path name might be wrong somewhere, what would be the next step towards a fix for this?

[spencer-lunarg]

@Izhido no problem, can you confirm you are still having issues with the latest 1.3.280 SDK version
https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases/tag/vulkan-sdk-1.3.280.0

[Izhido]

I can confirm that the issue is still present in 1.3.280:

[spencer-lunarg]

ok, I should be able to find time this week to take a look at this again and try to get a solution, thanks for pinging this again

[spencer-lunarg]

@Izhido I am looking at your error message

java.lang.UnsatisfiedLinkError: Unable to load native library "/data/app/~~eBT-ClU8d8rElTi09Pz4Rw==/com.my_namespace.my_app-9oOtV6cikWELznRSV6vYYA==/base.apk!/lib/arm64-v8a/libmy_app_native.so": dlopen failed: library "D:/Projects/MyApp/app/src/main/cpp/../../../../android-binaries-vulkan-sdk-1.3.268.0/arm64-v8a/libVkLayer_khronos_validation.so" not found: needed by /data/app/~~eBT-ClU8d8rElTi09Pz4Rw==/com.my_namespace.my_app-9oOtV6cikWELznRSV6vYYA==/base.apk!/lib/arm64-v8a/libmy_app_native.so in namespace classloader-namespace

And the

dlopen failed: library "D:/Projects/MyApp/app/src/main/cpp/../../../../android-binaries-vulkan-sdk-1.3.268.0/arm64-v8a/libVkLayer_khronos_validation.so" not found:

Makes it seem like the issue is you just have the path wrong in your building of your app and not an issue of the VVL binary itself

So for a normal app that has the layers built inside of it, if you go %ANDROID_SDK_ROOT%\cmdline-tools\latest\bin\apkanalyzer.exe files list YourApp.apk you should see

/lib/arm64-v8a/libVkLayer_khronos_validation.so

[spencer-lunarg]

as explained in the previous comment, from the error message, the issue seems to be the path is looking for a Windows path and not a valid Android path

Closing as this doesn't seem like an issue with the Android binary of VVL

[Izhido]

I apologize in advance if what I'm doing right now is in bad taste.

I believe, however, that I now have evidence to support the argument that the problem is currently in the build process of the library itself.

This was hinted to me by https://stackoverflow.com/questions/47279824/android-ndk-dlopen-failed , after checking for some other people that might have a similar problem.

Indeed, by running the readelf command mentioned there in the 1.3.236 and the 1.3.283 versions of the library, this is what I get:

The absence of the SONAME entry in the new library actually explains why the old library works, while the new one doesn't.
And the fix for that is provided in that same article.

Do you think you could reconsider giving this a new chance?

[Izhido]

UPDATE:
I went and added to the source code of 1.3.283, in /CMakeLists.txt, the following:

...
if(${CMAKE_CXX_COMPILER_ID} MATCHES "(GNU|Clang)")
    ...
    add_link_options(-Wl,-soname=libVkLayer_khronos_validation.so)
    ...
elseif(MSVC)
...

And I was able to confirm that the newly compiled library works fine.

(I'm pretty sure that is not the correct way, in this project, of adding the missing linking flags. I'll let you determine the right place for it.)

[spencer-lunarg]

@Izhido thanks for keep looking into this

Can you make an PR with your purpose fix (I want to see what worked for you)

if what works for you still works on our CI machine (and my local devices) then it is a win-win and we can merge it in

[Izhido]

#8116 <--- this is the one.

Took me a while to write a proper commit message, but I think I finally managed to do it :D .

Please, let me know if anything happens.

[Izhido]

To double-confirm: the newest changes by @spencer-lunarg fixed the issue!

(I also discovered, with this, that there is a component called Vulkan Loader that is generating some validation errors. But that will be for another team, another time.)

Thank you so much for your effort!

[ghost]

The absence of the SONAME entry in the new library actually explains why the old library works, while the new one doesn't.

That doesn't fully explain the situation. Why does it work in LunarG CI but not on your local environment? That issue you linked to is relevant for shared libraries you link to. But you don't link to the validation layers. It's opened via the vulkan loader via dlopen. So soname shouldn't matter AFAIK.

[Izhido]

I noticed there was one recent release (1.3.290), which still exhibits the same behavior.
Was the change not approved for release, after all?
Just wanted to know if that's what happened.

[spencer-lunarg]

sorry, there were concerns on the #8124 and I didn't have time to look into if they are valid or not... going to ping someone who is working more with Android to help confirm this/push it through now... sorry for the delay (I'm not a build expert by any means 😢 )

[DanAlbert]

@lzhido: how did you build your app? It looks like you linked the validation layer libraries directly. I think that's the problem. Vulkan loads those at runtime as long as that layer (VK_LAYER_KHRONOS_validation) is requested in your VkInstanceCreateInfo. You need to package them according to https://developer.android.com/ndk/guides/graphics/validation-layer#prebuilt-binaries, but your C++ build doesn't need to link against those libraries. See https://vulkan-tutorial.com/en/Drawing_a_triangle/Setup/Validation_layers.

[Izhido]

@DanAlbert : Thank you, so much, for the (definitive) tip!

I was indeed linking the validation layers directly to the project. After removing the linking, and when I copied the libs to the jniLibs folder, as stated in the Android docs you posted above, the problem disappeared, completely.

Now I'm a bit afraid of what else I've been missing regarding setup on my project - I think I need to review the existing docs a bit more thoroughly.

But, for now, this means my original request is not needed anymore. Feel free to close the PR, as it is no longer relevant for the validation layers in Android.

Also, thanks everyone for your patience on this issue!

[DanAlbert]

Glad I could help. You may want to subscribe on #8167, which would let you do this even more simply.