Apktool 3 Brainstorming (Remove aapt1)

[iBotPeaches]

I've been noticing that is getting incredibly difficult for myself to keep up with AOSP aapt/aapt2 on three different platforms. Its becoming a nightmare to just build it, because Android/AOSP/Google keep making a basic build-tools build incredibly difficult.

You'd think the tools group that repo offers would be what we want, but nah. You need to clone every since repo, but thankfully can take advantage of of shallow clones and only current branches.

Manually patching aapt or aapt2 with little knowledge of c++ and the ever evolving AOSP build system is killing me, just to be frank and honest. I either rely on contributors and then have to find my way around understand patches or struggle for months for the simplest of patches.

So here I want to brainstorm about Apktool 3, where I'm thinking about completely removing aapt1 from the tool. This means everything would be funneled through AAPT2. This would allow 1 path to patch and remove restrictions that folks are commonly reporting bugs for.

Not to say this journey would be easy in any regard of patching AAPT2, since its still unfamiliar terrain, but I'll take 1 set of patches vs double.

This would allow me to focus the current areas of improvement in aapt2 that are apktool specific:

• Pure static build of aapt2 for all platforms (seems bugged on mac)
• Investigate keeping platformBuildVersionName and platformBuildVersionCode unchanged
• Investigate keeping compileSdkVersion and compileSdkCodename unchanged

Related bugs:

• [BLOCKER - 2.5.1] AAPT not usable on macOS #2477
• compileSdkVersion value changed in AndroidManifest.xml file #2030
• PlatformBuildVersion information changes unexpectedly #1909

Opinions?

[IgorEisberg]

Yeah, scrapping AAPT is the right move. AAPT2 has everything AAPT had, and much more than that.
I use AAPT2 exclusively for a while now (with some of my own tweaks of course, to be able to build ID 1, removed some resource naming restrictions, and a few others).
The results are far better than I had with AAPT, especially the API versioning (like -v3), which is retained for the directories, but stripped from resources.arsc when not essential. This allows me to recreate an APK as identical as possible to the original APK, which prevents invalidating overlay APKs when the APK is supposed to be a system app. AAPT can't do that and forced me to recompile the relevant overlays as well to fix the inconsistency. In any case, there is nothing to lose by scrapping AAPT.

[MrIkso]

As for me, it is worth abandoning AAPT\AAPT2 altogether. Applications have long gone beyond the scope of AOSP and after compilation of resources, different post-processing of resources occurs. With the arrival of split apk, normal recompilation of resources became impossible.
What to do? There is a way out - to compile the resources with your own jvm compiler. What are the advantages:

• the ability to collect a resource table with many packages
• collect resources of varying complexity, since if the resource was not able to be decompiled correctly, there will be a raw value, which will then be written to the table.
• full control over compilation, no error binding from outside.

Also, there are already projects that can collect xml and arsc without aapt.
Creating axml
https://github.com/codyi96/xml2axml
Creating arsc
https://github.com/MCMrARM/arscwriter
Edirors arsc
https://github.com/Tencent/matrix/tree/master/matrix/matrix-android/matrix-arscutil
https://github.com/google/android-arscblamer
https://github.com/cmeiyuan/ArscEditor

And here is a project that tests resources. Contains good utility classes to convert values to binary format and parse resources to use pseudo compilation (testing resources)
https://github.com/robolectric/robolectric/

Well, and also, Android Studio itself also knows how to compile resources in the proto format. This project maybe suitable for understanding how to write a parser for decompiled resources then build xml and arsc, not proto.
https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-main:build-system/aaptcompiler/src/main/java/com/android/aaptcompiler/

The time has come to change the concept of apktool, because the further, the more applications are moving away from the canons of AOSP and what can aapt.

[iBotPeaches]

This is an interesting idea. I just don't know if I'm ready to inline a good chunk of code for resources building. Though, I will peek at those resources and links you gave to do some research.

[yanbuyu]

add some links, I hope it is useful:
https://github.com/yanbuyu/AndroidResProguard
https://github.com/bluesky466/ResourcesParser
https://www.jianshu.com/p/c2e5677053d6

[pashamcr]

@iBotPeaches you can use https://github.com/pashamcr/AAPTS, build for each platform takes 5 minutes, you can also create a fork, add a build for macOS, unify everything into one project, and only support the files needed to build aapt's:

https://android.googlesource.com/platform/build/soong/+archive/refs/heads/master/cc/libbuildversion.tar.gz
https://android.googlesource.com/platform/external/expat/+archive/refs/heads/master.tar.gz
https://android.googlesource.com/platform/external/fmtlib/+archive/refs/heads/master/include/fmt.tar.gz
https://android.googlesource.com/platform/external/googletest/+archive/refs/heads/master/googletest/include/gtest.tar.gz
https://android.googlesource.com/platform/external/libcxx/+archive/refs/heads/master.tar.gz
https://android.googlesource.com/platform/external/libcxxabi/+archive/refs/heads/master.tar.gz
https://android.googlesource.com/platform/external/libpng/+archive/refs/heads/master.tar.gz
https://android.googlesource.com/platform/external/protobuf/+archive/refs/heads/master/src/google/protobuf.tar.gz
https://android.googlesource.com/platform/external/protobuf/+archive/refs/heads/master/config.tar.gz
https://android.googlesource.com/platform/external/zlib/+archive/refs/heads/master.tar.gz
https://android.googlesource.com/platform/frameworks/base/+archive/refs/heads/master/cmds/idmap2/libidmap2_policies/include/idmap2.tar.gz
https://android.googlesource.com/platform/frameworks/base/+archive/refs/heads/master/libs/androidfw.tar.gz
https://android.googlesource.com/platform/frameworks/base/+archive/refs/heads/master/tools/aapt.tar.gz
https://android.googlesource.com/platform/frameworks/base/+archive/refs/heads/master/tools/aapt2.tar.gz
https://android.googlesource.com/platform/frameworks/native/+archive/refs/heads/master/include/android.tar.gz
https://android.googlesource.com/platform/system/core/+archive/refs/heads/master/libcutils.tar.gz
https://android.googlesource.com/platform/system/core/+archive/refs/heads/master/libsystem/include/system.tar.gz
https://android.googlesource.com/platform/system/core/+archive/refs/heads/master/libutils.tar.gz
https://android.googlesource.com/platform/system/incremental_delivery/+archive/refs/heads/master/incfs/util.tar.gz
https://android.googlesource.com/platform/system/libbase/+archive/refs/heads/master.tar.gz
https://android.googlesource.com/platform/system/libziparchive/+archive/refs/heads/master.tar.gz
https://android.googlesource.com/platform/system/logging/+archive/refs/heads/master/liblog.tar.gz

[iBotPeaches]

Okay, I've boiled this down to some items I'm focusing for 3.0

• AAPT1 is gone
• Tests (that are feasible) are ported from AAPT1 to AAPT2 for no lose of testing
• Parameters are maintained as best as possible, but CLI library is replaced
• Dummy parameters are configurable to not write (since aapt1 restriction is gone)
• Add parameter for disassembling AndroidManifest without resources (leaving resIds if they reference internal resources to then block recompiling via conf)
• Linter added to code to enforce CS
• Make API use of the library first class citizen
• Support mapping files to safe names for obfuscation tricks & reserved filenames
• Ignoring bundles, still only focusing Apks
• Investigating if Apktool can easily merge splits back to universal

For the reasons

• Dropping aapt/aapt2 entirely is not sustainable - too much knowledge and effort that is not here with myself
• Optimizing aapt build to be fast and scoped to only needed AOSP repos is also not sustainable with knowledge of AOSP with myself