From e75edaec62110bd8d3ea321b80c9a0b572482161 Mon Sep 17 00:00:00 2001 From: Prasanth Pulavarthi Date: Thu, 12 Oct 2023 19:08:09 -0700 Subject: [PATCH] Update pytorch-on-the-edge.html --- blogs/pytorch-on-the-edge.html | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/blogs/pytorch-on-the-edge.html b/blogs/pytorch-on-the-edge.html index a44d1187fad3b..c05bdf18a75ff 100644 --- a/blogs/pytorch-on-the-edge.html +++ b/blogs/pytorch-on-the-edge.html @@ -271,11 +271,11 @@

Speech recognition with Whisper on mobile

Train a model to recognize your voice on mobile

-

ONNX Runtime can also take a pre-trained model and adapt it to data that you provide. It can do this on the edge, on mobile specifically where it is easy to record your voice, access your photos and other personalized data. Importantly, your data does not leave the device during training.

+

ONNX Runtime can also take a pre-trained model and adapt it to new data. It can do this on the edge - on mobile specifically where it is easy to record your voice, access your photos and other personalized data. Importantly, your data does not leave the device during training.

For example, you can train a PyTorch model to recognize just your own voice on your mobile phone, for authentication scenarios.

-

The PyTorch model is obtained from HuggingFace, and extra layers are added to perform the speaker classification.

+

The PyTorch model is obtained from HuggingFace in your development environment, and extra layers are added to perform the speaker classification:


 from transformers import Wav2Vec2ForSequenceClassification, AutoConfig
@@ -288,7 +288,7 @@ 
Train a model to recognize your voice on mobile
-

The model and other components necessary for training (a loss function to measure the quality of the model and an optimizer to instruct how the weights are adjusted during training) are exported with the ONNX Runtime training Python API.

+

The model and other components necessary for training (a loss function to measure the quality of the model and an optimizer to instruct how the weights are adjusted during training) are exported with ONNX Runtime Training:


 artifacts.generate_artifacts(
@@ -301,7 +301,7 @@ 
Train a model to recognize your voice on mobile

 )
-

This set of artifacts is now ready to be loaded by the mobile application, shown here as iOS Swift code. Within the application, a number of samples of the speaker’s audio are provided to the application and the model is trained with the samples.

+

This set of artifacts is now ready to be loaded by the mobile app, shown here as iOS Swift code. The app asks the user for samples of their voice and the model is trained with the samples.


 func trainStep(inputData: [Data], labels: [Int64]) throws  {
@@ -323,10 +323,8 @@ 
Train a model to recognize your voice on mobile

 
                                     
Where to next?

 
-                                    
In this article we’ve shown why you would run PyTorch models on the edge and what aspects to consider. We also shared several examples with code that you can use for running state-of-the-art PyTorch models on the edge with ONNX Runtime. We also showed how ONNX Runtime was built for performance and cross-platform execution, making it the ideal way to run PyTorch models on the edge. You may have noticed that we didn’t include a Llama2 example even though ONNX Runtime is optimized to run it. That’s because the amazing Llama2 model deserves its own article, so stay tuned for that!

-
-                                    
You can read more about how to run your PyTorch models on the edge here: https://onnxruntime.ai/docs/

-
+                                    
In this article we’ve shown why you would run PyTorch models on the edge and what aspects to consider. We also shared several examples with code that you can use for running state-of-the-art PyTorch models on the edge with ONNX Runtime. We also showed how ONNX Runtime was built for performance and cross-platform execution, making it the ideal way to run PyTorch models on the edge. Have fun running PyTorch models on the edge with ONNX Runtime!

+                                    
You may have noticed that we didn’t include a Llama2 example even though ONNX Runtime is optimized to run it. That’s because the amazing Llama2 model deserves its own article, so stay tuned for that!