This website is able to take a stream url or an mp4 file and then transcribe what was said in Japanese and then translate it into English. It does this by splitting the file or stream into chunks and sends these to OpenAI's whisper model for transcribing and then it sends this transcription to DeepL for translation. Presently it'll send a request at every few seconds to get the latest things that were said after an initial short buffer period.
It's certainly not perfect but hopefully it's enough for English speakers to understand the context around streams.
Currently with beta testers so please wait for now, if you'd like to give it a try message me on twitter or something! Probably will be waiting for #20 until I post the link publicly incase of api abuse.
- You'll need bun, ffmpeg and streamlink to be available in your command line.
- Also setup a redis server somewhere and grab the connection url for it. I like railway!
- Next grab API keys for OpenAi + DeepL. Then clone the repo to your local machine and once downloaded, copy
.env.exampleto.envfilling it out with your OpenAi+DeepL api keys and redis url from step 2. - Run
bun imstallwithin the repo folder to install more dependencies. - Now you should be able to run
bun run fakerto start the dev servers which is enough to get started. Once started you should be able to go to http://localhost:3000 to see the site. - If you wish to run the site with actual transcriptions and translations then run
bun run dev.
OpenAI gives you $5 to use for the first few months after signing up. I think you can just keep creating accounts to get more. You'll be spending $0.006 a minute with OpenAI so a 1 hour stream will cost you ~$0.36 meaning you can watch just under 14 hours of streams with the free $5 you are given.
DeepL has a free allowance of 500,000 characters a month. It seems plenty fine!