Publish onnxruntime-extensions Builds for Java to Maven Central

[HichemMaiza]

Hello Guys, Can we publish onnxruntime-extensions to maven central ? This addition would significantly benefit the Java community, allowing users to easily integrate it into their projects through Maven or Gradle dependencies. Currently, adding onnxruntime to a Java project is straightforward with the following Maven dependency:

<!-- https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime -->
<dependency>
    <groupId>com.microsoft.onnxruntime</groupId>
    <artifactId>onnxruntime</artifactId>
    <version>1.16.1</version>
</dependency>

To further enhance the experience, we may add the following dependency

<!-- https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime -->
<dependency>
    <groupId>com.microsoft.onnxruntime</groupId>
    <artifactId>onnxruntime-extensions</artifactId>
    <version>0.9.0</version>
</dependency>

I have personally built version 0.9.0 and tested it with OnnxRuntime version 1.16.0 across multiple models in the Microsoft/Olive repository. You can find my builds on GitHub https://github.com/HichemMaiza/onnxruntime-extensions-java.

This addition to Maven Central would greatly simplify the integration of onnxruntime-extensions into Java projects and open up new possibilities for the Java community.

Thank you for considering this proposal. Please let me know if you need any further information.

[EmergentOrder]

+1

[HossamAmer12]

@HichemMaiza I am facing a similar issue. I looked up your github link but does not seem understand clearly the steps to include onnxruntime-extensions in my Kotlin+Java project. For example, I do not have pom.xml in my project

I added this in my build.gradle.kts:

implementation("com.hichemmaiza.onnx:onnxruntime-extensions-java:0.10.0-SNAPSHOT")

I see this error:

• What went wrong:
  Execution failed for task ':app:compileKotlin'.
  Error while evaluating property 'filteredArgumentsMap' of task ':app:compileKotlin'.
  Could not resolve all files for configuration ':app:compileClasspath'.
  > Could not find com.hichemmaiza.onnx:onnxruntime-extensions-java:0.10.0-SNAPSHOT.
  Searched in the following locations:
  - https://repo.maven.apache.org/maven2/com/hichemmaiza/onnx/onnxruntime-extensions-java/0.10.0-SNAPSHOT/maven-metadata.xml
  - https://repo.maven.apache.org/maven2/com/hichemmaiza/onnx/onnxruntime-extensions-java/0.10.0-SNAPSHOT/onnxruntime-extensions-java-0.10.0-SNAPSHOT.pom
  Required by:
  project :app

Can you please further explain the steps?

[HossamAmer12]

I installed mvn3.9.5 from here , ran mvn clean install from your github repo
and saw the following:

Still not sure how to import this.

[HossamAmer12]

I added in my build.gradle.kts the following:

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
    // maven local
    mavenLocal()

}

dependencies {
implementation("com.hichemmaiza.onnx:onnxruntime-extensions-java:0.10.0-SNAPSHOT")
}

Not able yet to import it in my Kotlin+Java code to be able to use the library.
I tried this import com.hichemmaiza.onnx.*; . Tells me unresolved reference to hichemmaiza

Any help?

[Craigacp]

The name used in the gradle dependency is not the package name. You should still use import ai.onnxruntime.extensions.OrtxPackage;.

[HossamAmer12]

I did the import @Craigacp.. but that's the error I got:

/home/hossamamer/workspace/Emoji_workspace/Android-emoji/app/src/main/kotlin/EmojiPredictor.kt: (11, 34): Unresolved reference: OrtxPackage

Any clue?

[HichemMaiza]

@HossamAmer12 as @Craigacp already mentioned, you need to do

import ai.onnxruntime.extensions.OrtxPackage; 
// ... 
OrtSession.SessionOptions  sessionOptions = new OrtSession.SessionOptions() ;
sessionOptions.registerCustomOpLibrary(OrtxPackage.getLibraryPath());

[HossamAmer12]

@HichemMaiza I did this import but I still get the error. Also did the steps above

[HichemMaiza]

@HossamAmer12 Thank you for your feedback, may I ask what OS are you using ?

[HossamAmer12]

@HichemMaiza I am using WSL 2.0 on Windows OS.

[wenbingl]

check here: https://github.com/microsoft/onnxruntime-extensions/tree/main/tutorials/demo4j, you may need to manully copy the jar file to app libs folder.

Publish jar file to Maven Central could be done in the late releases if the resource allows us to do so.

[HossamAmer12]

@wenbingl I looked up the link and found this comment:
copy JAR package from $REPO/out/$OS/RelWithDebInfo/java/build/libs/onnxruntime-extensions-${VERSION}.jar into app/libs folder.

I could not find or locate the directory above. Any clue? or any chance you can share a jar file for 0.8 ORT-extensions?

[wenbingl]

@wenbingl I looked up the link and found this comment: copy JAR package from $REPO/out/$OS/RelWithDebInfo/java/build/libs/onnxruntime-extensions-${VERSION}.jar into app/libs folder.

I could not find or locate the directory above. Any clue? or any chance you can share a jar file for 0.8 ORT-extensions?

The package was built from source, https://github.com/microsoft/onnxruntime-extensions/blob/main/java/README.md

I just verified the build is still OK, and shared the JAR file for experiment here: https://1drv.ms/u/s!AgQZTAPsze9AiM8lA6AfPF6bL7803Q?e=8Yt9UZ

[HossamAmer12]

Thanks @wenbingl
I created a libs folder in my project and pasted your JAR file into this folder. But I still get this import error. What do I need to do to ensure that JAR file is properly imported in my project?

[Craigacp]

Is your grade build set to look for dependencies in the libs folder? That's not the default behaviour for regular non-android gradle builds.

[HossamAmer12]

@Craigacp How can I do that?

[Craigacp]

https://discuss.gradle.org/t/how-to-include-local-jar-file-in-gradle/37744/2

[HossamAmer12]

My build.gradle.kts should follow the kotlin way to include.

I followed this link:
https://stackoverflow.com/questions/54166069/how-do-you-add-local-jar-file-dependency-to-build-gradle-kt-file

The file with
import ai.onnxruntime.extensions.OrtxPackage was OK

As soon as I add this statement, it does not build:

 val sessionOptions = OrtSession.SessionOptions()
/* Register the custom ops from onnxruntime-extensions */
sessionOptions.registerCustomOpLibrary(OrtxPackage.getLibraryPath());

error:

(39, 5): Expecting member declaration

[Craigacp]

That sounds like a Kotlin syntax problem rather than any issue with the library. Is the line registering the ops library inside a function or init block?

[wenbingl]

OrtxPackage

check here: https://github.com/microsoft/onnxruntime-inference-examples/blob/45f8c0cdad97eefc1623b0044e9a4ac961343725/mobile/examples/super_resolution/android/app/src/main/java/ai/onnxruntime/example/superresolution/MainActivity.kt#L42

But I am not a Kotlin user, only see the difference between yours and theirs.

[Craigacp]

It would be nice to get Java builds going for this, the stable diffusion sample code I released earlier this week tells users to compile this repo from source and copy the binary in which isn't particularly user friendly - https://github.com/oracle-samples/sd4j.

[HichemMaiza]

@Craigacp Thank you for your contribution; I completely agree !!
I used ONNX to infer the Whisper model, and I didn't want to upload it before finding a solution to onnxruntime-extensions. Asking Java developers to compile the repo means we've already lost 90% of them.

[HossamAmer12]

Just following up on this issue, has there been any updates?
@wenbingl wenbingl

[wenbingl]

Just following up on this issue, has there been any updates? @wenbingl wenbingl

it's in the plan, and am looking for the resource and it could be done in the next month.

[HichemMaiza]

@wenbingl Great news !

[HossamAmer12]

@wenbingl has there been any updates on this?

[wenbingl]

@sayanshaw24 is working on a pipeline to compile JNI for multiple platforms into one Jar file now.

[HossamAmer12]

@wenbingl @sayanshaw24 any pointer how to use the new change?

[wenbingl]

@wenbingl @sayanshaw24 any pointer how to use the new change?

It will be available in Maven repo on the next ort-extensions 0.11 release, which will be early next month

[karllessard]

Just by curiosity, any visibility on when 0.11 will then be released? Thank you

[wenbingl]

Just by curiosity, any visibility on when 0.11 will then be released? Thank you

The release team is working on that, and the packages will be public in a couple of days.

[karllessard]

Thanks @wenbingl. Playing at building the JAR myself meanwhile, I realize that the current build conflicts with the onnxruntime release, as both are exported as Java module under the same name: com.microsoft.onnxruntime, as we can see here: https://github.com/microsoft/onnxruntime-extensions/blame/8d8670f7e87e9bb456b4b017879e8e4bb5aedfe3/java/build.gradle#L116

Hence, the compilation fails if I have a modular project depending on both com.microsoft.onnxruntime:onnxruntime and com.microsoft.onnxruntime:onnxruntime-extensions. Will that be fixed for the release?

[Craigacp]

I made a PR to fix the module name #730.

[Craigacp]

The release on Maven Central of ORT extensions v0.11.0 contains x64 binaries for Windows, macOS and Linux, but ORT v1.18.0 contains binaries for arm64 macOS and Linux as well as the x64 binaries. I think support for building the arm64 binaries has been added to the project (at least on the build system side, not sure about the CI), is it possible to get them folded into the next release of ORT extensions on Maven Central?

[david-sitsky]

I've mentioned this on #764, but com.microsoft.onnxruntime:onnxruntime-extensions:0.11.0 is broken. The embedded onnxruntime_extensions4j_jni.dll file is missing the RegisterCustomOps function, which means it can't be used at all on Windows.

The RegisterCustomOps function is there for Linux in libonnxruntime_extensions4j_jni.so, but for some reason this file is a whooping 176M in size! The equivalent .so in the 0.10.0 Nuget package is 5.1MB in size.

Can we please fix this? This is a blocker for Windows projects.

[Craigacp]

The 0.12.4 release on Maven Central is still missing macOS arm64 (and Linux arm64, though that's less relevant for my usecase). Is it possible to get that binary added to the jar in the next release?