compatibility with ML models - hugging face candle rust

[flutter-painter]

Hi !
Is rinf compatible with https://github.com/huggingface/candle ?
It could help distribute ML local/privacy first apps built in flutter
I dare ask because I tried using candle with flutter_rust_bridge and it failed
I understand the two are very different and rinf should be much more stable #195

[temeddix]

If you can run huggingface models in Rust, you should be able to run them in Rinf or flutter_rust_bridge. However without additional information, it's not easy for me to provide more details. Wish you luck!

[flutter-painter]

Thank you for your reply. I reckon you could promote rinf with a desktop app like maid https://github.com/Mobile-Artificial-Intelligence/maid easing access to all ML models available in candle. I tried running MarianMT using the bridge, but it did not work. This is my main use case : user inputs text in flutter and the rust model response is displayed in a text field below. It could then be used in translator app such as biyi https://github.com/lijy91 I also wonder if you know how much additional weight (Mo) using rust/ rinf adds to the binary app. If it adds too much, ONNX might be safer for mobile use cases. I

…

________________________________ From: Donghyun Kim ***@***.***> Sent: Monday, September 9, 2024 6:07:22 PM To: cunarist/rinf ***@***.***> Cc: Pierre Gancel ***@***.***>; Author ***@***.***> Subject: Re: [cunarist/rinf] compatibility with ML models - hugging face candle rust (Discussion #414) If you can run huggingface models in Rust, you should be able to run them in Rinf or flutter_rust_bridge. However without additional information, it's not easy for me to provide more details. Wish you luck! — Reply to this email directly, view it on GitHub<#414 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADMBPBV4YZ7QR6NGQM3722TZVXBTVAVCNFSM6AAAAABNM57P32VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJZGMZDONA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[temeddix]

Thank you very much for your opinion :) It's happy to see people finding Rinf useful. I'll definitely consider promiting Rinf with the idea that you've wrote here.