Skip to content

ente-io/onnxruntime_flutter

 
 

Repository files navigation

OnnxRuntime Plugin

pub package

Overview

Flutter plugin for OnnxRuntime with FFI provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.

Key Features

  • Multi-platform Support for Android and iOS
  • Flexibility to use any Onnx Model.
  • Acceleration using multi-threading.
  • Similar structure as OnnxRuntime Java and C# API.
  • Inference speed is not slower than native Android/iOS Apps built using the Java/Objective-C API.
  • Run inference in different isolates to prevent jank in UI thread.

Getting Started

In your flutter project add the dependency:

dependencies:
  ...
  onnxruntime: x.y.z

Usage example

Import

import 'package:onnxruntime/onnxruntime.dart';

Initializing environment

OrtEnv.instance.init();

Creating the Session

final sessionOptions = OrtSessionOptions();
const assetFileName = 'assets/models/test.onnx';
final rawAssetFile = await rootBundle.load(assetFileName);
final bytes = rawAssetFile.buffer.asUint8List();
final session = OrtSession.fromBuffer(bytes, sessionOptions!);

Performing inference

final shape = [1, 2, 3];
final inputOrt = OrtValueTensor.createTensorWithDataList(data, shape);
final inputs = {'input': inputOrt};
final runOptions = OrtRunOptions();
final outputs = await _session?.runAsync(runOptions, inputs);
inputOrt.release();
runOptions.release();
outputs?.forEach((element) {
  element?.release();
});

Releasing environment

OrtEnv.instance.release();

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Dart 91.6%
  • C++ 8.2%
  • C 0.1%
  • Ruby 0.1%
  • Swift 0.0%
  • Kotlin 0.0%