Integrating WhisperKit into an existing pipeline - pass in CMSampleBuffers?

[vade]

hi there.

Thanks for WhisperKit, it looks awesome!

Im new to the API, and am curious how I might go about having whisper kit consume CMSampleBuffers of audio (which I could request in a specific format if I knew it).

Thanks!

[ZachNagengast]

Hi @vade, good question - do you have CMSampleBuffers in your code already that you are trying to pass in to WhisperKit? If so, the easiest way to do that would be to convert them into a single [Float] array at 16khz sample rate, and pass that into try await whisperKit.transcribe(audioArray: floatArray). The library will handle chunking and resizing to fit the model, so the array can be arbitrary length (as long as it fits into memory).

Also curious about how you're using CMSampleBuffers, I've been interested in some lower level apis for audio capture from microphones. The models require single channel 16khz float audio array, but there is no limitation other than that.