diff --git a/.gitignore b/.gitignore index 5823b44e..988879bd 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,8 @@ /.idea /android/verification/ + +# Ignore all generated Maven local repository files and folders +/android-release-support/maven/pom.xml +/android-release-support/maven/rustls/rustls-platform-verifier/**/ +/android-release-support/maven/rustls/rustls-platform-verifier/maven-metadata-local.xml \ No newline at end of file diff --git a/Cargo.lock b/Cargo.lock index a4eea190..60a4df44 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -313,6 +313,7 @@ dependencies = [ "once_cell", "rustls", "rustls-native-certs", + "rustls-platform-verifier-android", "rustls-webpki", "security-framework", "security-framework-sys", @@ -320,6 +321,10 @@ dependencies = [ "winapi", ] +[[package]] +name = "rustls-platform-verifier-android" +version = "0.1.0" + [[package]] name = "rustls-webpki" version = "0.101.4" diff --git a/Cargo.toml b/Cargo.toml index 11a96577..c29f147e 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,4 +1,7 @@ [workspace] -members = ["rustls-platform-verifier"] +members = [ + "android-release-support", + "rustls-platform-verifier", +] resolver = "2" diff --git a/README.md b/README.md index 57e29f9d..8338d496 100644 --- a/README.md +++ b/README.md @@ -47,64 +47,55 @@ component must be included in your app's build to support `rustls-platform-verif #### Gradle Setup `rustls-platform-verifier` bundles the required native components in the crate, but the project must be setup to locate them -automatically and correctly. +automatically and correctly. These steps assume you are using `.gradle` Groovy files because they're the most common, but everything +is 100% applicable to Kotlin script (`.gradle.kts`) configurations too with a few replacements. -Firstly, create an [init script](https://docs.gradle.org/current/userguide/init_scripts.html) in your Android -Gradle project, with a filename of `init.gradle`. This is generally placed in your project's root. In your project's `settings.gradle`, add these lines: +Inside of your project's `build.gradle` file, add the following code and Maven repository definition. If applicable, this should only be the one "app" sub-project that +will actually be using this crate at runtime. With multiple projects running this, your Gradle configuration performance may degrade. -```groovy -apply from: file("./init.gradle"); -// Cargo automatically handles finding the downloaded crate in the correct location -// for your project. -def veifierProjectPath = findRustlsPlatformVerifierProject() -includeBuild("${verifierProjectPath}/android/") -``` - -Next, the `rustls-platform-verifier` external dependency needs to be setup. Open the `init.gradle` file and add the following: -`$PATH_TO_DEPENDENT_CRATE` is the relative path to the Cargo manifest (`Cargo.toml`) of any crate in your workspace that depends on `rustls-platform-verifier` -from the location of your `init.gradle` file. - -Alternatively, you can use `cmdProcessBuilder.directory(File("PATH_TO_ROOT"))` to change the working directory instead. +`$PATH_TO_DEPENDENT_CRATE` is the relative path to the Cargo manifest (`Cargo.toml`) of any crate in your workspace that depends on `rustls-platform-verifier` from +the location of your `build.gradle` file: ```groovy -ext.findRustlsPlatformVerifierProject = { - def cmdProcessBuilder = new ProcessBuilder(new String[] { "cargo", "metadata", "--format-version", "1", "--manifest-path", "$PATH_TO_DEPENDENT_CRATE" }) - def dependencyInfoText = new StringBuffer() +import groovy.json.JsonSlurper - def cmdProcess = cmdProcessBuilder.start() - cmdProcess.consumeProcessOutput(dependencyInfoText, null) - cmdProcess.waitFor() +// ...Your own script code could be here... - def dependencyJson = new groovy.json.JsonSlurper().parseText(dependencyInfoText.toString()) - def manifestPath = file(dependencyJson.packages.find { it.name == "rustls-platform-verifier" }.manifest_path) - return manifestPath.parent +repositories { + // ... Your other repositories could be here... + maven { + url = findRustlsPlatformVerifierProject() + metadataSources.artifact() + } } -``` -This script can be tweaked as best suits your project, but the `cargo metadata` invocation must be included so that the Android -implementation source can be located on disk. +String findRustlsPlatformVerifierProject() { + def dependencyText = providers.exec { + it.workingDir = new File("../") + commandLine("cargo", "metadata", "--format-version", "1", "--manifest-path", "$PATH_TO_DEPENDENT_CRATE/Cargo.toml") + }.standardOutput.asText.get() -If your project often updates its Android Gradle Plugin versions, you should additionally consider setting your app's project -up to override `rustls-platform-verifier`'s dependency versions. This allows your app to control what versions are used and avoid -conflicts. To do so, advertise a `versions.path` system property from your `settings.gradle`: - -```groovy -ext.setVersionsPath = { - System.setProperty("versions.path", file("your/versions/path.toml").absolutePath) + def dependencyJson = new JsonSlurper().parseText(dependencyText) + def manifestPath = file(dependencyJson.packages.find { it.name == "rustls-platform-verifier-android" }.manifest_path) + return new File(manifestPath.parentFile, "maven").path } +``` -setVersionsPath() +Then, wherever you declare your dependencies, add the following: +```groovy +implementation "rustls:rustls-platform-verifier:latest.release" ``` -Finally, sync your gradle project changes. It should pick up on the `rustls-platform-verifier` Gradle project. It should finish -successfully, resulting in a `rustls` group appearing in Android Studio's project view. -After this, everything should be ready to use. Future updates of `rustls-platform-verifier` won't need any maintenance beyond the -expected `cargo update`. +Cargo automatically handles finding the downloaded crate in the correct location for your project. It also handles updating the version when +new releases of `rustls-platform-verifier` are published. If you only use published releases, no extra maintenance should be required. + +These script snippets can be tweaked as best suits your project, but the `cargo metadata` invocation must be included so that the Android +implementation part can be located on-disk. #### Proguard If your Android application makes use of Proguard for optimizations, its important to make sure that the Android verifier component isn't optimized -out because it looks like dead code. Proguard is unable to see any JNI usage, so your rules must manually opt into keeping it. THe following rule +out because it looks like dead code. Proguard is unable to see any JNI usage, so your rules must manually opt into keeping it. The following rule can do this for you: ```text -keep, includedescriptorclasses class org.rustls.platformverifier.** { *; } diff --git a/admin/RELEASING.md b/admin/RELEASING.md new file mode 100644 index 00000000..dd5cf4ba --- /dev/null +++ b/admin/RELEASING.md @@ -0,0 +1,34 @@ +# How-to release `rustls-platform-verifier` + +This document records the steps to publish new versions of the crate since it requires non-trivial preparation and ordering +that needs to be remembered due to the Android component's distribution. + +## Steps + +1. Update main crate's version in `rustls-platform-verifier/Cargo.toml`. +2. If any non-test changes have been made to the `android` directory since the last release: + 1. Update Android artifact version in `android-release-support/Cargo.toml` + 2. Bump dependency version of the Android support crate in `rustls-platform-verifier/Cargo.toml` to match the new one + 3. Commit version increase changes on the release branch + * We typically name these branches `rel-xxx` where `xxx` is the major version. + * We typically leave these branches around for future maintenance releases. + 4. Run `ci/package_android_release.sh` in a UNIX compatible shell + 5. (Optional) `cargo publish -p rustls-platform-verifier-android --dry-run --alow-dirty` + + * `--allow-dirty` is required because we don't check-in the generated Maven local repository at this time. + 6. (Optional) Inspect extracted archive to ensure the local Maven repository artifacts are present + 1. Un-tar the `rustls-platform-verifier-android-*.crate` file inside of `target/package`. + 2. Verify `maven/rustls/rustls-platform-verifier` contains a single `*.RELEASE` directory and that contains a `.aar` file. + 3. (Optional) If the releaser has an external Gradle project that uses the configuration from the README, paste the path to the + unzipped package's `Cargo.toml` as a replacement for the `manifestPath` variable. Run a Gradle Sync and observe everything works. + 7. Publish the Android artifacts' new version: `cargo publish -p rustls-platform-verifier-android --alow-dirty` +3. Commit main crate's version increase on the release branch +4. Publish the main crate's new version: `cargo publish -p rustls-platform-verifier` + * Do **not** use `--allow-dirty` for the main crate. Only the Android component requires it and a dirty workspace elsewhere is an error. + +See the Rustls repo [RELEASING] guidance for more information (e.g. on best practices for creating a GitHub release with a changelog). + +[RELEASING]: https://github.com/rustls/rustls/blob/main/RELEASING.md diff --git a/android-release-support/Cargo.toml b/android-release-support/Cargo.toml new file mode 100644 index 00000000..da7063b4 --- /dev/null +++ b/android-release-support/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "rustls-platform-verifier-android" +version = "0.1.0" +description = "The internal JVM support component of the rustls-platform-verifier crate. You shouldn't depend on this directly." +repository = "https://github.com/rustls/rustls-platform-verifier" +license = "MIT OR Apache-2.0" +edition = "2021" + +# Explicitly include the Maven local repository for the Android component. +# While not checked into the repository, it is generated for releases and other contexts. +include = [ + "src/*", + "maven/pom.xml", + "maven/rustls/rustls-platform-verifier/**/", + "maven/rustls/rustls-platform-verifier/maven-metadata-local.xml", +] + +[dependencies] diff --git a/android-release-support/maven/rustls/rustls-platform-verifier/.gitkeep b/android-release-support/maven/rustls/rustls-platform-verifier/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/android-release-support/pom-template.xml b/android-release-support/pom-template.xml new file mode 100644 index 00000000..cbc86eb9 --- /dev/null +++ b/android-release-support/pom-template.xml @@ -0,0 +1,10 @@ + + + 4.0.0 + rustls + rustls-platform-verifier + $VERSION + aar + The internal JVM support component of the rustls-platform-verifier Rust crate + \ No newline at end of file diff --git a/android-release-support/src/lib.rs b/android-release-support/src/lib.rs new file mode 100644 index 00000000..97fb79ad --- /dev/null +++ b/android-release-support/src/lib.rs @@ -0,0 +1,75 @@ +//! # rustls-platform-verifier-android +//! +//! This crate is an implementation detail of the actual [rustls-platform-verifier](https://github.com/rustls/rustls-platform-verifier) crate. +//! +//! It contains no Rust code and is solely intended as a convenient delivery mechanism for the supporting Kotlin code that the main crate +//! requires to perform TLS certificate validation using Android's APIs. +//! +//! Other crates should not directly depend on this crate in any way, as nothing about it is considered stable and it is probably useless elsewhere. +//! +//! ## Details +//! +//! Note: Everything in this section is subject to change at any time. Semver may not be followed. +//! +//! ### Why? +//! +//! It was the best middle ground between several tradeoffs. The important ones, in priority order, are: +//! - Automatically keeping component versions in sync +//! - Allowing well-tested and well-known `cargo` dependency management patterns to apply everywhere +//! - Providing a smooth developer experience as an Android consumer of `rustls-platform-verifier` +//! +//! Firstly, what alternatives are available for distributing the component? The other two known are source distribution in some form (here, it will be through crates.io) +//! and Maven Central. Starting with the first, its become infeasible due to toolchain syncing requirements. If the Android component is +//! built as part of the host app's Gradle build, then it becomes subject to any Gradle or Android Gradle Plugin incompatibilities/requirements. In practice this means +//! the AGP version between this project and the main application have to match all the time. Sometimes this works, but it becomes challenging/unfeasible +//! during yearly toolchain/SDK upgrades and is not maintainable long term. Note that this is the _only_ option in this section which retains compatibility +//! with Cargo's Git dependency patching. +//! +//! Next, Maven Central. This is considered the standard way of distributing public Android dependencies. There are two downsides to this +//! approach: version synchronization and publishing overhead. Version syncing is the hardest part: There's not a good way to know what version +//! a crate is that doesn't hurt the Cargo part of the build or damage functionality. So instead of making assumptions at runtime, we would need to do +//! clunky and manual version counting with an extra error case. Less importantly, the admin overhead of Maven Central is non-zero so its good to avoid +//! if possible for such a small need. +//! +//! It is also worth calling out a third set of much worse options: requiring users to manually download and install the Android component +//! on each update, which magnifies the version syncing problem with lots of user overhead and then deleting the component outright. A rewrite +//! could be done with raw JNI calls, but this would easily be 3x the size of the existing implementation and require huge amounts of `unsafe` +//! to review then audit. +//! +//! ### The solution +//! +//! The final design was built to avoid the pitfalls the previous two options mentioned. To build it, we rely on CI and packaging scripts to build +//! the Android component into a prebuilt AAR file before creating a release. Next, a [on-disk Maven repository](https://maven.apache.org/repositories/local.html) +//! is hosted inside of this repository. Only the unchanging file structure of it is kept checked-in, to avoid churn. The remaining parts are filled in +//! during the packaging/release process, before being included in `cargo package` via an `include` Cargo.toml directive. Finally, once the repository has had +//! its artifacts added the crate containing the Maven repository is published to crates.io. Then, the main crate ensures it's downloaded when an Android target +//! is compiled via a platform-specific dependency. +//! +//! On [the Gradle side](https://github.com/rustls/rustls-platform-verifier/tree/main#gradle-setup), we include a very small snippet of code for users to include in their `settings.gradle` file +//! to dynamically locate the local maven repository on disk automatically based off Cargo's current version of it. The script is configuration cache friendly and +//! doesn't impact performance either. When the script is run, it finds the cargo-cached download of the crate and tells Gradle it can find the Android component there +//! when it gets sourced into the hosting application's build tree. +//! +//! Assuming a properly configured Gradle project, the slow (~500ms) script should only run once per Gradle sync while the `android-release-support` crate +//! remains untouched. This is due to the configuration cache previously mentioned and is what ensures performance on-par with a "normal" Maven repository. +//! Upon any version updates (semver, Git refs, etc), the change will be detected as-intended by Gradle, break the cache, and the project will update the dependency reference to the new AAR file. +//! +//! ### Precompiled artifacts? +//! +//! For some, the notion of shipping something pre-compiled with an existing source distribution might seem incorrect, or insecure. However in this specific case, +//! putting aside the fact shipping Kotlin code doesn't work (see above), there are many reasons this isn't the case: +//! - Shipping pre-compiled artifacts is normal in the Java ecosystem. Maven Central and other package repositories do the same thing and serve `.jar` downloads. +//! - Those not using Android will never download the pre-compiled AAR file. +//! - The artifacts are incredibly easy to reproduce given an identical compilation toolchain. +//! - The artifacts are not native executables, or raw `.jar` files, so they can't be accidentally executed on a host system. +//! +//! ## Summary +//! +//! In summary, the selected distribution method avoids most of the previous pitfalls while still balancing a good experience for `cargo` and Gradle users. Some of its +//! positive properties include: +//! - Full compatibility with Cargo's dependency management, including Git patching[^1] +//! - No version checking or synchronization required +//! - Painless and harmless to integrate into an Android app's build system +//! - Low maintenance for the main crate maintainers' +//! +//! [^1]: The Git reference being used must have the local maven repository built and checked-in first. diff --git a/android/settings.gradle b/android/settings.gradle index e4416705..e00d83fe 100644 --- a/android/settings.gradle +++ b/android/settings.gradle @@ -12,19 +12,9 @@ dependencyResolutionManagement { mavenCentral() } - // We use a version catalog for two reasons: - // 1. Ease of dependency management - // 2. Supporting having our versions overridden by a larger project including us - // as a composite build. This lets versions stay in sync when they would otherwise become - // incompatible. Examples of this include AGP, where an actual app might have a newer one - // then this library (which doesn't need to). - // - // This works by first trying to read global property that a parent module could advertise - // and then falling back to our local definitions. In combination, both project-local tasks - // still work as intended and use as a library in a full application. versionCatalogs { libs { - from(files(System.getProperty("versions.path", "gradle/libraries.versions.toml"))) + from(files("gradle/libraries.versions.toml")) } } } diff --git a/ci/package_android_release.sh b/ci/package_android_release.sh new file mode 100755 index 00000000..91b06eac --- /dev/null +++ b/ci/package_android_release.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash + +# This script's purpose is to automate the build + packaging steps for the pre-compiled Android verifier component. +# It works with template files and directories inside the `android-release-support/` part of the repository to setup +# a Maven local repository and then add the pre-compiled AAR file into it for distribution. The results of this packaging +# are then included by `cargo` when publishing `rustls-platform-verifier-android`. + +set -euo pipefail + +if ! type mvn > /dev/null; then + echo "The maven CLI, mvn, is required to run this script." + echo "Download it from: https://maven.apache.org/download.cgi" + exit 1 +fi + +version=$(grep -m 1 "version = " android-release-support/Cargo.toml | tr -d "version= " | tr -d '"') + +echo "Packaging v$version of the Android support component" + +pushd ./android + +./gradlew assembleRelease + +popd + +artifact_name="rustls-platform-verifier-release.aar" + +pushd ./android-release-support + +artifact_path="../android/rustls-platform-verifier/build/outputs/aar/$artifact_name" + +# Ensure no prior artifacts are present +git clean -dfX "./maven/" + +cp ./pom-template.xml ./maven/pom.xml + +# This sequence is meant to workaround the incompatibilites between macOS's sed +# command and the GNU command. Referenced from the following: +# https://stackoverflow.com/questions/5694228/sed-in-place-flag-that-works-both-on-mac-bsd-and-linux +sed -i.bak "s/\$VERSION/$version/" ./maven/pom.xml +rm ./maven/pom.xml.bak + +mvn install:install-file -Dfile="$artifact_path" -Dpackaging="aar" -DpomFile="./maven/pom.xml" -DlocalRepositoryPath="./maven/" diff --git a/rustls-platform-verifier/Cargo.toml b/rustls-platform-verifier/Cargo.toml index 8099dc5a..9403a64b 100644 --- a/rustls-platform-verifier/Cargo.toml +++ b/rustls-platform-verifier/Cargo.toml @@ -4,19 +4,11 @@ version = "0.1.0" authors = ["ComplexSpaces ", "1Password"] description = "rustls-platform-verifier supports verifying TLS certificates in rustls with the operating system verifier" keywords = ["tls", "certificate", "verification", "os", "native"] -repository = "https://github.com/1Password/rustls-platform-verifier" +repository = "https://github.com/rustls/rustls-platform-verifier" license = "MIT OR Apache-2.0" edition = "2021" rust-version = "1.64.0" -exclude = [ - "android/.run", - "android/gradle/**", - "android/gradle*", - "android/settings.gradle", - "android/src/androidTest", -] - [lib] name = "rustls_platform_verifier" # Note: The `cdylib` specification is for testing only. The shared library @@ -49,6 +41,7 @@ once_cell = "1.9" webpki = { package = "rustls-webpki", version = "0.101", features = ["alloc", "std"] } [target.'cfg(target_os = "android")'.dependencies] +rustls-platform-verifier-android = { path = "../android-release-support", version = "0.1.0" } jni = { version = "0.19", default-features = false } webpki = { package = "rustls-webpki", version = "0.101", features = ["alloc", "std"] } once_cell = "1.9"