Compile to smali fails on IntelliJ 2021.1

[auermich93]

There seems to be some issue related to IntelliJ 2021.1 and the java2smali plugin:

org.jf.dexlib2.DexFileFactory$DexFileNotFoundException: Tracer.dex does not exist
	at org.jf.dexlib2.DexFileFactory.loadDexFile(DexFileFactory.java:83)
	at org.jf.dexlib2.DexFileFactory.loadDexFile(DexFileFactory.java:61)
	at org.ollide.java2smali.Dex2SmaliHelper.disassembleDexFile(Dex2SmaliHelper.kt:24)
	at org.ollide.java2smali.DexCompiler$onProjectBuildComplete$1.run(DexCompiler.kt:67)
	at com.intellij.openapi.command.WriteCommandAction.lambda$runWriteCommandAction$4(WriteCommandAction.java:347)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.lambda$doRunWriteCommandAction$1(WriteCommandAction.java:143)
	at com.intellij.openapi.application.impl.ApplicationImpl.runWriteAction(ApplicationImpl.java:959)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.lambda$doRunWriteCommandAction$2(WriteCommandAction.java:141)
	at com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:216)
	at com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:182)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.doRunWriteCommandAction(WriteCommandAction.java:150)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.run(WriteCommandAction.java:117)
	at com.intellij.openapi.command.WriteCommandAction.runWriteCommandAction(WriteCommandAction.java:347)
	at com.intellij.openapi.command.WriteCommandAction.runWriteCommandAction(WriteCommandAction.java:335)
	at org.ollide.java2smali.DexCompiler.onProjectBuildComplete(DexCompiler.kt:66)
	at org.ollide.java2smali.DexCompiler.access$onProjectBuildComplete(DexCompiler.kt:22)
	at org.ollide.java2smali.DexCompiler$run$1.invoke(DexCompiler.kt:26)
	at org.ollide.java2smali.DexCompiler$run$1.invoke(DexCompiler.kt:22)
	at org.ollide.java2smali.DexCompiler$buildModule$1.accept(DexCompiler.kt:46)
	at org.ollide.java2smali.DexCompiler$buildModule$1.accept(DexCompiler.kt:22)
	at org.jetbrains.concurrency.AsyncPromise$onSuccess$1.accept(AsyncPromise.kt:84)
	at org.jetbrains.concurrency.AsyncPromise$onSuccess$1.accept(AsyncPromise.kt:16)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
	at org.jetbrains.concurrency.AsyncPromise.setResult(AsyncPromise.kt:149)
	at com.intellij.task.impl.ProjectTaskManagerImpl$ResultConsumer.lambda$notify$1(ProjectTaskManagerImpl.java:351)
	at com.intellij.openapi.application.TransactionGuardImpl.runWithWritingAllowed(TransactionGuardImpl.java:218)
	at com.intellij.openapi.application.TransactionGuardImpl.access$200(TransactionGuardImpl.java:21)
	at com.intellij.openapi.application.TransactionGuardImpl$2.run(TransactionGuardImpl.java:200)
	at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:781)
	at com.intellij.openapi.application.impl.ApplicationImpl.lambda$invokeLater$4(ApplicationImpl.java:319)
	at com.intellij.openapi.application.impl.FlushQueue.doRun(FlushQueue.java:84)
	at com.intellij.openapi.application.impl.FlushQueue.runNextEvent(FlushQueue.java:133)
	at com.intellij.openapi.application.impl.FlushQueue.flushNow(FlushQueue.java:46)
	at com.intellij.openapi.application.impl.FlushQueue$FlushNow.run(FlushQueue.java:189)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:776)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:746)
	at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:969)
	at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:839)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$8(IdeEventQueue.java:449)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computePrioritized(CoreProgressManager.java:808)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$9(IdeEventQueue.java:448)
	at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:781)
	at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:496)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

The issue was not present on earlier versions of IntelliJ.

[auermich93]

I resolved the issue on my own, I have updated to Java15 since this is a pure java application except that single class
I need to convert to smali. Downgrading to Java8 resolves the issue.

[auermich93]

I was playing around with different java versions and I noticed that Java11 is working as well under certain circumstances.
It is working within IntelliJ. However, when I try to convert a .class file compiled with Java11 to a dex file on the command line, it only works when using d8 instead of dx. In addition, this might only work since my java code that is converted to smali doesn't use language features introduced with a recent Java versions. Thus my question to you @ollide:
Why is Java15 not working? Is it a limitation of d8/dx or java2smali? Any chance that it is supported in the near future?

[ollide]

@auermich93 thanks for the follow-up (and the ping). I can probably take a closer look tomorrow.

[auermich93]

Unfortunately, the issue is more severe than I thought. It was a false conclusion that Java11 is working out-of-the-box within IntelliJ.
It highly depends on what you actually build and somehow java2smali re-uses a cached result if the build fails (but no exception is shown to the user). I have tested the following within IntelliJ and Java11:

I have created a new class containing a single field and a single method that prints something via System.out.println().
That's really plain old Java. When I click on 'Build -> Compile to Smali' this works as expected.
Then, I have added to this class another method containing a lambda expression (stream api). If I try now to convert it to smali,
it doesn't fail, but returns the cached, i.e. the previous, smali class to me without any hint about an error. Only when I do a
complete rebuild or delete the respective .dex file within the /build/classes/ folder, the above reported stack trace is shown to me. @ollide, is this intended behaviour?
The good thing is that Java8 is working, ofc only stuff that is expected to be working with Java8. There might be a solution to this, but this depends on whether java2smali is using d8, see https://jakewharton.com/androids-java-9-10-11-and-12-support/.
I haven't tested it yet. In my case, I will move the code to a dedicated sub module and tell gradle to use Java8 for it.

[ollide]

I was able to reproduce your findings and you're right with the cached result that is being used by java2smali.

The plugin operates in 3 sequential steps:

1. Trigger a module/project build for the given file (here, IDEA compiles the .java file into one or multiple .class file)
2. Run dx.jar to transform the .class file(s) into a .dex file
3. Run baksmali to transform the .dex file into the final .smali file

Step 1 fails, if IDEA is unable to compile the project / module. Step 3 fails whenever there's no dex file available from step 2 (this is the error from your first message). Unfortunately step 2 seems to fail silently, so baksmali may run on a previously emitted .dex file instead of throwing an error.

First, I'm going to fix the silent failing of step 2. Next I'll take a look what I can do to better support new language features. Are you actually running java2smali's smali code on Android devices? I expect it to be rather hard to produce executable code in all scenarios.

[auermich93]

Nice that you can confirm my results. Yes, the code is actually running on Android devices. My java application is instrumenting an APK. In particular, I inject a custom tracer class, which is nothing else than a broadcast receiver. I am using dexlib2 under the hood, which allows me to read directly from smali files and emits dalvik bytecode.

[ollide]

First, I'm going to fix the silent failing of step 2. Next I'll take a look what I can do to better support new language features.

@auermich93 The first part is done, so the plugin no longer falls back to cached dex files.

I've also played a bit with improved language support and there might be a small but powerful fix for this: Explicitly setting an API level. dx also asks for this but before #25 this information got lost because of the silent failures.

I introduced the API level change on another branch, see here. It would be amazing if you could try this out before I publish a new release.

Here's a fresh build of that branch java2smali-2.1.0-SNAPSHOT.zip, of course you could also build it from the sources.

Thanks again for the detailed report and investigation.

[auermich93]

Thanks for investing your time into this issue. I tested the following within IntelliJ and Java15 as compiler:

I created dummy methods and included in each a specific language feature. In particular, I tested Java8 (Optional, Stream), Java11 (var keyword, isBlank()), Java12 (Collectors.teeing()), Java13 (new switch case) and Java15 (new instance of check).
The convertion into smali worked flawlessly for any language feature. Then, I actually tried to test things on an android emulator targeting API Level 28. I started with the Java8 features, those worked. However, when I included the Java11 features, a verification error appears: [0x16] Call site #5 too few arguments for bootstrap method: 4 < 5

This corresponds to the smali code:
invoke-custom {v2}, call_site_2("makeConcatWithConstants", (Z)Ljava/lang/String;, "IsBlank: \u0001")@Ljava/lang/invoke/StringConcatFactory;->makeConcatWithConstants(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;

This invoke-custom instruction seems to be the 'tool' to implement those new language features in Android, it was also used for other features like the StreamAPI but also for other regular method invocations. I believe that Java11 is not compatible with API 28, but Java8 is. I gonna try out tommorrow whether API 30 (Android 11) can handle those language features yet.

@ollide: Regarding determining the API version, you could use dexFile.getOpcodes().api; if your code makes use of the DexFile class of dexlib2. This automatically determines the API version based on the used opcodes.

Btw, I didn't test the silent failure fix.

[auermich93]

I tested the same thing on API 30. It's not anymore the verification error, but a warning followed by a NoSuchMethodError:
Accessing hidden method Ljava/lang/invoke/LambdaMetafactory;->metafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite; (blacklist, linking, denied)

2021-04-26 10:03:49.107 7187-7187/? E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.zola.bmi, PID: 7187
    java.lang.NoSuchMethodError: No static method metafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite; in class Ljava/lang/invoke/LambdaMetafactory; or its super classes (declaration of 'java.lang.invoke.LambdaMetafactory' appears in /apex/com.android.art/javalib/core-oj.jar)

And I have to correct above statements about Java8. I included logs and noticed that only the first part was executed successfully and the remaining instructions fail. In particular, the Stream-API stuff of my test method didn't work:

    private static void java8() {

        Optional<String> opt = Optional.of("test");

        if (opt.isPresent()) {
            System.out.println(opt.get()); // this log is printed
        }

        List<Integer> ints = Arrays.asList(10, 12, 4, 3).stream().filter(i -> i > 10).collect(Collectors.toList());
        System.out.println("Numbers: " + ints); // this one not and leads to the above warning/exception
    }

@ollide: Are you checking whether dx is on the PATH or are you always using a pre-shipped dx tool? This might also have
an effect about which API version is supported in the general case (not in my case). I gonna try to test the stuff directly
on the command line with both dx and d8 and see if I can get anything running there. Results follow later.

[auermich93]

I made some progress already. The basis is the following Test.java file:

import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;

public class Test {

    private static void java11() {
        var s = "test";
        System.out.println(s);
        System.out.println("IsBlank: " + s.isBlank());
    }


    private static void java8() {

        Optional<String> opt = Optional.of("test");

        if (opt.isPresent()) {
            System.out.println(opt.get());
        }

        List<Integer> ints = Arrays.asList(10, 12, 4, 3).stream().filter(i -> i > 10).collect(Collectors.toList());
        System.out.println("Numbers: " + ints);
    }

    public static void main(String[] args) {
        java8();
        java11();
    }
}

You have to compile this file with Java11 (Java15 is not recognized by d8), i.e. javac Test.java. Then, open an emulator (API 30) and run the following sequence of commands:

$ANDROID_HOME/build-tools/30.0.2/d8.bat Test.class --intermediate --output output.zip --lib $ANDROID_HOME/platforms/android-30/android.jar
adb push output.zip /data/local (on Windows you have to drop the first slash -> data/local)
adb shell dalvikvm -cp /data/local/output.zip Test (on Windows you have to drop the first slash -> data/local/output.zip)

The output is as follows:

test
Numbers: [12]
test
IsBlank: false

If you drop the --lib option for the d8 command, it will show you the warnings and some stuff about desugaring, but it still works. Check https://developer.android.com/studio/command-line/d8 for more information about d8.

I tried the same thing with the dx tool. However, Java11 is not recognized by dx. Thus, I removed the java11() method and re-compiled Test.java with Java8. Then, I ran the following commands:

$ANDROID_HOME/build-tools/30.0.2/dx.bat --dex --output=classes.dex --min-sdk-version=30 Test.class
zip output.zip classes.dex
adb push output.zip /data/local (on Windows you have to drop the first slash -> data/local)
adb shell dalvikvm -cp /data/local/output.zip Test (on Windows you have to drop the first slash -> data/local/output.zip)

However, the output is as follows:

test
Exception in thread "main" java.lang.BootstrapMethodError: Exception from call site #0 bootstrap method
        at Test.java8(Test.java:24)
        at Test.main(Test.java:29)
Caused by: java.lang.ClassCastException: Bootstrap method returned null
        ... 2 more

As we can observe, the Optional class is supported, but the Stream-API stuff fails. I couldn't find any option for dx that
supports desugaring like d8 does. @ollide: How do you invoke the dx tool, because java2smali works with class files that were compiled with Java11 or even Java15, while I get the following error on the command line:

PARSE ERROR:
unsupported class file version 59.0
...while parsing Test.class
1 error; aborting

[ollide]

@auermich93 thanks again for your thorough tests. I'll try to comment on all relevant aspects, let me know if I miss anything.

Regarding determining the API version, you could use dexFile.getOpcodes().api; if your code makes use of the DexFile class of dexlib2. This automatically determines the API version based on the used opcodes.

This is good to know, but won't help here as java2smali creates the dex file itself (and thereby dictates the opcodes version) and later reverses the dex file using the same API version.

Are you checking whether dx is on the PATH or are you always using a pre-shipped dx tool?

I'm using a self-contained version of dx, you can see it here. The main reason for this is that java2smali is used in many different IDEs (Android Studio, IDEA Ultimate, IDEA Community Edition) across several versions and it would be hard to a) locate the external dx and b) maintain compatibility with all dx versions. Also the setup and usage of the plugin would be more prone to errors.

How do you invoke the dx tool, because java2smali works with class files that were compiled with Java11 or even Java15, while I get the following error on the command line

dx is invoked by the Class2DexHelper which runs within the plugin's execution context inside the IDE. This either means you tried an older version of dx which might not be compatible with newer bytecode versions or running dx with an older Java version. So far I've been only running tests with Java 8 and Java 11 (bytecode 55.0) and both worked flawlessly.

Language Support

Now let's switch to the most important part: How can we support all new language features or more precisely, all language features that modern Android versions support?

Preface: I'm by no means an expert on this topic, so there will be some inaccuracies or even misinformation.

In the old days, Android was very slow with the support of new Java language features and new features were only introduced with new Android versions and without any backward compatibility. This comes down to the opcode versions of the dex bytecode (produced by dx) and Dalvik, the dex bytecode runtime on the device. Dalvik was replaced with ART (Android Runtime) in Android Lollipop (API 21) but still consumed the dex bytecode that was produced by dx. With API 28 Android introduced the d8 tool that supports desugaring and deprecated dx shortly after. In fact, dx has been removed from the SDK last week.

Desugaring just means that the syntactic sugar of new language features is replaced with standard expressions that are functionally identical. So if we introduce d8 into java2smali we should be able to support all of the new language features.

When I've first looked at d8 I was worried to do so because it has a dependency on the android.jar which must be passed with the --lib parameter:

--lib Specifies the path to the android.jar of your Android SDK. This flag is required when compiling bytecode that uses Java 8 language features.

But in your tests you were able to use Java 8 language features and run them on a device without specifying it?

If you drop the --lib option for the d8 command, it will show you the warnings and some stuff about desugaring, but it still works.

If we can run d8 without the android.jar this could be changed quite easily. When it is indeed required, I could potentially define an optional dependency on the Android SDK in the plugin's manifest and locate the android.jar for d8. For IDEs that don't have the Android plugin installed, a fallback to dx would still work as it does today.

[auermich93]

@ollide: The attached screenshot should prove that d8 works without specifying the --lib parameter. As mentioned,
it shows couple of warnings but compiles flawlessly.

The latest version of d8 can also read Java15 (in contrast to previous versions, see above comments), but shows the following warning:

$ $ANDROID_HOME/build-tools/31.0.0-rc3/d8.bat Test.class --intermediate --output output.zip --lib $ANDROID_HOME/platforms/android-30/android.jar
Warning in Test.class:
  One or more classes has class file version >= 56 which is not officially supported.

However, the latest version of dx fails with class files of Java11 and Java15 and I don't understand why, since it works within your plugin:

Michael@Michael-Laptop MINGW64 /e/Work/test
$ javap -verbose Test | grep "major"
  major version: 55

Michael@Michael-Laptop MINGW64 /e/Work/test
$ $ANDROID_HOME/build-tools/31.0.0-rc3/dx.bat --dex --output=classes.dex --min-sdk-version=30 Test.class

PARSE ERROR:
unsupported class file version 55.0
...while parsing Test.class
1 error; aborting

Android-Studio says that 31.0.0-rc3 is the latest build tools version and you are using the same version according to
https://github.com/ollide/intellij-java2smali/blob/develop/lib/dx-31.0.0-rc3.jar. Do you see any mistake in here?
It can be that ghosts are following me. 😟 While I was writing this post, I was trying to run d8 of version 30.0.2 as in the previous posts within the same shell. While it succeeded the first time, it failed the other times complaining that d8.bat is missing. And I said WTF is going on here. It was really missing in the folder. The only possible explanation to me is that I opened Android-Studio during this time and it was synchronizing files automatically. I get really punished by these kinds of errors lately....

[ollide]

@auermich93 D8 desugaring indeed seems to work without any --library arguments.

I've introduced D8 on the feature branch here: 698c3f7.
Here's a snapshot build with D8 you can try out: java2smali-2.1.0-SNAPSHOT.zip

When running the plugin on your Test class, the plugin will emit 3 smali files and none of them contains any invoke-custom instructions:

• Test.smali: The smali code of the original Test class
• Test$$ExternalSyntheticBackport0.smali: A desugared backport of Java 11's String#isBlank method
• Test$$ExternalSyntheticLambda1.smali: A backport for the lambda instruction

I think it will be necessary to test the D8 build a little more before I can remove dx entirely and release a new plugin version. Please let me know, if you can actually run the produced smali code on a device.

[auermich93]

@ollide If I run baksmali on the classes.dex file, I obtain also these three smali files. As you can observe from my previous post,
I am actually running these files on an emulator. I push the output.zip containing the classes.dex on the emulator (Nexus 5 API 30) and invoke through adb shell dalvikvm -cp /data/local/outzip.zip Test the main method of the Test class.

One question is what's the dex version of the generated classes.dex file. Independent, whether I use the Java11 or Java15 compiled Test.class file, the output is as follows:

According to https://android.googlesource.com/platform/dalvik/+/master/dx/src/com/android/dex/DexFormat.java#67, dex version 035 corresponds to API level 13 and earlier, thus the generated dex code should run on pretty all devices.

But still one urgent question for me is why java2smali can handle Java15 bytecode:

As shown in my previous post, dx can't handle Java15. I even picked the dx.jar from your repository (https://github.com/ollide/intellij-java2smali/tree/develop/lib) and this confirms my results:

$ java -jar ~/Downloads/dx-31.0.0-rc3.jar --dex --output=classes.dex --min-sdk-version=30 Test.class

PARSE ERROR:
unsupported class file version 59.0
...while parsing Test.class
1 error; aborting

So there is some magic involved in the background of java2smali, otherwise I can't explain why java2smali can handle
Java15. How is your tool building the Java file? Which compiler is used?