Executorch Android Build Docs seem out-of-date

[JCodeShelver]

I am using executorch for deploying a pytorch model to an Android app I am co-developing. I am unsure how else to give my feedback, so I am opening an issue.

Following the documentation at https://pytorch.org/executorch/main/getting-started-setup.html and then https://pytorch.org/executorch/main/demo-apps-android.html, skipping over the code for dl3 exporting and QNN use, since I am using neither, I would repeatedly encounter issues with building the executorch library via cross-compiling (code at https://pytorch.org/executorch/main/demo-apps-android.html#xnnpack.

Note: I am using Windows 11, and have installed and run these executorch commands in Ubuntu WSL. Using the NDK installed by using SDK Manager in Android Studio (I selected the latest, which happens to be v28, which is in beta currently) installs, appropriately, the windows version of the NDK.

However, the cmake commands do not like that, and end up looking for the C and C++ compilers in {NDKPATH}/toolchains/llvm/linux-x86_64/, which is non-existent.

I eventually found that by installing the main branch executorch in my WSL distro, via
git clone -b viable/strict https://github.com/pytorch/executorch.git,
combined with r27 of the NDK (latest stable release), which I installed the LINUX ZIP of from the GitHub at https://github.com/android/ndk/releases by moving it to WSL (I installed via web browser in Windows), and unzipping.

Using the WSL, linux variant of the stable NDK, with the main branch, worked. I also specified the ANDROID_PLATFORM option to CMake when building the executorch library and android extension, by passing the flags present at #5387 (comment), and setting the ANDROID_PLATFORM appropriately for my app (the minimum supported SDK version).
When building the extension, I used the same flags as the executorch docs, but also added the -DANDROID_PLATFORM=android-<min-sdk-number-here>, then ran the build of the extension the same way as the tutorial.

I am unsure how many of these steps are necessary, but for cross-compiling for Android, from WSL, with Android Studio installed in Windows, this is how I got it to work. I can give an even more thorough step-by-step if really needed/wanted.

[kirklandsign]

Hi @JCodeShelver thank you for the feedback.

Just want to make sure I understand correctly, you mentioned, we need these changes

1. ${ANDROID_NDK}/build/cmake/android.toolchain.cmake doesn't give a correct compiler path in CMake
2. Need to specify ANDROID_PLATFORM in cmake invocation.

Are these required to make it work?

[JCodeShelver]

I would wager if my Android Studio were installed in my WSL distro, then when I installed the NDK from it, it would have installed the Linux toolchain. Because I am running the cmake command from WSL (linux on Windows), the toolchain file will try to look in the ndk at ${ANDROID_NDK}/toolchains/llvm/linux-x86_64, which would be correct if the NDK was the linux version (which I had to manually install, as the version Android Studio installed had ${ANDROID_NDK}/toolchains/llvm/windows-x86_64).

I am pretty sure this applies to people using WSL more than anything. The current executorch website documentation says that when installing and setting up the repo and dev environment on Windows to use WSL with the supported linux options. In the Android Demo App tutorial, it does say to install Android Studio, but does NOT make it clear that you should either:
a) do this in WSL, or
b) do the workaround I said with manually installing the Linux NDK.

ANDROID_PLATFORM might be necessary? I am unsure. I finally managed to get this to work after 3 days of red errors.

I'm looking at the examples/demo-apps/android/ExecuTorchDemo Android project to figure out how they interfaced with the exported library files, because the tutorial said that NativePeer.java (visible in the screenshot of the tutorial, but looking through git history, was removed months ago) interacts with it.

Digging more, there seems to be a "setup.sh" file in the included example demo app (executorch/examples/demo-apps/android/ExecuTorchDemo/setup.sh) that...I think automates the process? It has a Meta copyright at the top, and doesn't look to have logic for the WSL workaround.

[kirklandsign]

Sorry I'm not familiar with WSL and I never tried going through the documentation on WSL. To make sure, is it WSL 2 (basically a linux virtual machine)?

I will put do the workaround I said with manually installing the Linux NDK because it totally treats it as a Linux machine.

setup.sh is for linux and macos.

[JCodeShelver]

Also, there is a typo in the setup.sh script I found:
Line 18 reads:
-DEXECUTORCH_BUILD_EXTENSION_TESNOR=ON \
but should read:
-DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \

[kirklandsign]

Would you mind opening a PR for this?

[JCodeShelver]

Sorry I'm not familiar with WSL and I never tried going through the documentation on WSL. To make sure, is it WSL 2 (basically a linux virtual machine)?

I will put do the workaround I said with manually installing the Linux NDK because it totally treats it as a Linux machine.

setup.sh is for linux and macos.

Yep! WSL2, I happen to be running Ubuntu 20 or 22 or similar. It met the minimum cited on the website.

[JCodeShelver]

Sorry I'm not familiar with WSL and I never tried going through the documentation on WSL. To make sure, is it WSL 2 (basically a linux virtual machine)?

I will put do the workaround I said with manually installing the Linux NDK because it totally treats it as a Linux machine.

setup.sh is for linux and macos.

Yep! WSL2, I happen to be running Ubuntu 20 or 22 or similar. It met the minimum cited on the website.

[JCodeShelver]

Would you mind opening a PR for this?

What would I have in my PR?
I'm unfamiliar with bash scripting or editing CMake files.

[kirklandsign]

Would you mind opening a PR for this?

What would I have in my PR? I'm unfamiliar with bash scripting or editing CMake files.

I mean just fix this typo -DEXECUTORCH_BUILD_EXTENSION_TESNOR=ON \

[kirklandsign]

Yep! WSL2, I happen to be running Ubuntu 20 or 22 or similar

Then you can run the sh right?

[JCodeShelver]

Yep! WSL2, I happen to be running Ubuntu 20 or 22 or similar

Then you can run the sh right?

If it doesn't work without the ANDROID_PLATFORM variable specified, how do I fix that? I know Android BuildConfig build setting allows referencing Gradle stuff in Source code and the Manifest files, but I doubt bash would understand.

Is the goal that the setup.sh would be used as a build script that re-runs on each build (if changed) in Gradle?

[kirklandsign]

Actually the purpose of setup.sh here is to build the native library (so). Honestly I should update it to use the similar one as LlamaDemo (AAR). Right now we haven't uploaded the AAR (which contains JAR and so) to a repo like maven, so we need to build the AAR and use it locally. It's not really related to gradle itself, but just builds a dependency used by the app

[JCodeShelver]

If it was uploaded to Maven, would there be different versions like one with XNNPACK, one without, etc? Seems like too many options for one .so per configuration in Maven. Of course, the most common cases could be uploaded, and anything more custom would require building from source.

Also, if the .so is uploaded to Maven, is the only requirement by those using it in Android to just export a model to .pte?

[kirklandsign]

would there be different versions like one with XNNPACK, one without, etc?

That's something we need to consider, and thus we don't have it yet :)

Yes, expect the most common cases could be uploaded, and anything more custom would require building from source

Also, if the .so is uploaded to Maven, is the only requirement by those using it in Android to just export a model to .pte?

I think so

[kirklandsign]

We don't have a timeline for it yet.

[JCodeShelver]

Couldn't you just replace the setup.sh in the ExecuTorchDemo app with the onenin the LlamaDemo app?

[kirklandsign]

Couldn't you just replace the setup.sh in the ExecuTorchDemo app with the onenin the LlamaDemo app?

yes we can do that

[kirklandsign]

^ Needs to modify build.gradle as well though

[JCodeShelver]

What do I actually end up needing to integrate a .pte file with ny existing Android app? I have the .so files and android extension built from source, but if I go to use a .AAR, do I need to use fbjni like Llama does?

The AI model trained by one of my team members has the goal of, given a 224x224 3 channel image, output a number. I just need to be able to call that functionality in Kotlin from our app.

The LlamaDemo seems to have way more documentation in it, period. I think a lot less of it is applicable though, it uses things like fbjni and other files that seem to make using it a hassle.

[kirklandsign]

What do I actually end up needing to integrate a .pte file with ny existing Android app? I have the .so files and android extension built from source, but if I go to use a .AAR, do I need to use fbjni like Llama does?

AAR = JAR + so. You still need fbjni deps, in java lib dependency.

[kirklandsign]

I plan to update docs later, but the easiest for you is

sh build/build_android_llm_demo.sh

which builds the AAR, and then add the AAR to your deps such as

implementation("com.facebook.fbjni:fbjni:0.5.1")
implementation(files("libs/executorch.aar"))

[JCodeShelver]

This might be depressing, but did I compile the .so for nothing? It seems I did not need to figure out the toolchains and all that if I do not even end up using the .so.

[JCodeShelver]

I plan to update docs later, but the easiest for you is

sh build/build_android_llm_demo.sh

which builds the AAR, and then add the AAR to your deps such as

implementation("com.facebook.fbjni:fbjni:0.5.1")
implementation(files("libs/executorch.aar"))

You said to run the script at build/build_android_llm_demo.sh, but where is this folder?
It's in executorch/build/build_android_llm_demo.sh. I'm trying to see if there's a way I can just build with my current .so and other artifacts from successfully building the android extension. I'm trying to make as small of a footprint as possible with the Runtime and other things in the AAR.

[kirklandsign]

This might be depressing, but did I compile the .so for nothing? It seems I did not need to figure out the toolchains and all that if I do not even end up using the .so.

You compiled the so file which is zipped into the AAR file later. https://github.com/pytorch/executorch/blob/main/build/build_android_llm_demo.sh#L108-L121 is how you make the AAR