android app example with whisper_base_cpu_int8.onnx gives out of memory error

[suyash-narain]

I am using an aarch64 android mobile device using android 12 with 2GB RAM, 16GB ROM

when i use the android app example with the whisper_tiny_cpu_int8_model.onnx as provided, it works without issues. the model size if around 74MB.

When i replace this model with whisper_base_cpu_int8_model.onnx, which has a size of 140MB, it fails with out of memory error as it tries to access java heap memory instead of native memory.
All i did was change the model, i did not change the code anywhere. while running the app, i still have 500MB of free RAM available and still i get out of memory error.

the error log is below:

02-13 20:54:39.039 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:39.039 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:39.063 1973 2038 I le.whisperLoca: Alloc young concurrent copying GC freed 2084(104KB) AllocSpace objects, 2(1488KB) LOS objects, 3% free, 138MB/143MB, paused 137us,47us total 23.358ms
02-13 20:54:39.063 1973 2038 I le.whisperLoca: Forcing collection of SoftReferences for 133MB allocation
02-13 20:54:39.064 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:39.140 1973 2038 I le.whisperLoca: Alloc concurrent copying GC freed 399(27KB) AllocSpace objects, 2(134MB) LOS objects, 49% free, 3879KB/7758KB, paused 128us,39us total 75.682ms
02-13 20:54:41.492 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:41.492 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:41.507 1973 2038 I le.whisperLoca: Alloc young concurrent copying GC freed 49(47KB) AllocSpace objects, 0(0B) LOS objects, 4% free, 137MB/143MB, paused 148us,35us total 15.306ms
02-13 20:54:41.508 1973 2038 I le.whisperLoca: Forcing collection of SoftReferences for 133MB allocation
02-13 20:54:41.508 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:41.534 1973 2038 I le.whisperLoca: Alloc concurrent copying GC freed 738(47KB) AllocSpace objects, 0(0B) LOS objects, 4% free, 137MB/143MB, paused 298us,37us total 26.083ms
02-13 20:54:41.535 1973 2038 W le.whisperLoca: Throwing OutOfMemoryError "Failed to allocate a 140086368 byte allocation with 6291456 free bytes and 118MB until OOM, target footprint 150295640, growth limit 268435456" (VmSize 14958388 kB)
02-13 20:54:41.535 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:41.535 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:41.549 1973 2038 I le.whisperLoca: Alloc young concurrent copying GC freed 4(31KB) AllocSpace objects, 0(0B) LOS objects, 4% free, 137MB/143MB, paused 128us,40us total 13.928ms
02-13 20:54:41.550 1973 2038 I le.whisperLoca: Forcing collection of SoftReferences for 133MB allocation
02-13 20:54:41.550 1973 2038 I le.whisperLoca: Starting a blocking GC Alloc
02-13 20:54:41.575 1973 2038 I le.whisperLoca: Alloc concurrent copying GC freed 34(16KB) AllocSpace objects, 0(0B) LOS objects, 4% free, 137MB/143MB, paused 128us,40us total 25.049ms
02-13 20:54:41.576 1973 2038 W le.whisperLoca: Throwing OutOfMemoryError "Failed to allocate a 140086368 byte allocation with 6291456 free bytes and 118MB until OOM, target footprint 150294824, growth limit 268435456" (VmSize 14958388 kB)

how can I execute this? i should not be getting any memory error here. i generated the model using --no_audio_decoder option.

model is here: https://drive.google.com/file/d/1mrtEbq4PGcfppTVfGK2wMqRfniBivtB5/view?usp=sharing

[skottmckay]

microsoft/onnxruntime#19514