Lyrics/subtitles support

[anishbajpai014]

Are there any plans to add lyrics/subtitles support?

[barolo]

It's very easy to transcribe audio nowadays, and it's an important accessibility feature for hearing impaired folks.

[ansible42]

You can live caption using Android settings(settings -> sound ->live caption) It's not the best, but it's something.

I bet this could be leveraged. I would also love for saved snippets to be transcribed with a copy to clipboard and share option.

[anishbajpai014]

You can live caption using Android settings(settings -> sound ->live caption) It's not the best, but it's something.

I bet this could be leveraged. I would also love for saved snippets to be transcribed with a copy to clipboard and share option.

These work but from what I understand, it happens realtime and the text is delayed. If there is an api which allows us to do this in advance, I would love to try and integrate it.

I'm thinking more in terms of a spotify like synchronous text. Using something like on device whisper (by OpenAI) to transcribe in advance.

[ansible42]

That would for sure be better. I feel like the @PaulWoitaschek is trying to avoid outside API calls (sensible in my opinion) as much as possible. Running audio files through a LLM transcription service is going to get expensive real quick. Also opens up the issue of files which may or may not have been acquired through the secondary market shall we say.

[anishbajpai014]

Whisper is open source, has on device models. So everything will happen on device.

[ansible42]

Cool! I'll check that out

[barolo]

I know that there's at least one app that does exactly that, uses whisper [presumably] for on-device transcription. https://castleplayer.com/
Personally, as a first step I'd like to have support just for reading user supported synced subtitles [srt, webvtt, or something of this sort.]
There are dozens of whisper based tools to generate them, like https://github.com/jianfch/stable-ts, whisper-timestamped and so on.