How to specify the language of whisper for conversion to text?

[aef5748]

I have a Chinese voice file that I want to convert to text, but the ru language is always detected.

5:06AM INF Success ip=192.168.101.20 latency=19.601981559s method=POST status=200 url=/v1/audio/transcriptions
5:07AM INF Success ip=127.0.0.1 latency="28.387µs" method=GET status=200 url=/readyz
5:08AM DBG Request received: {"model":"whisper-medium-q5_0","language":"","translate":false,"n":0,"top_p":null,"top_k":null,"temperature":null,"max_tokens":null,"echo":false,"batch":0,"ignore_eos":false,"repeat_penalty":0,"repeat_last_n":0,"n_keep":0,"frequency_penalty":0,"presence_penalty":0,"tfz":null,"typical_p":null,"seed":null,"negative_prompt":"","rope_freq_base":0,"rope_freq_scale":0,"negative_prompt_scale":0,"use_fast_tokenizer":false,"clip_skip":0,"tokenizer":"","file":"","size":"","prompt":null,"instruction":"","input":null,"stop":null,"messages":null,"functions":null,"function_call":null,"stream":false,"mode":0,"step":0,"grammar":"","grammar_json_functions":null,"backend":"","model_base_name":""}
5:08AM DBG guessDefaultsFromFile: not a GGUF file
5:08AM DBG Audio file copied to: /tmp/whisper2830240388/chinese_sample_audio.mp3
5:08AM INF Loading model 'ggml-model-whisper-medium-q5_0.bin' with backend whisper
5:08AM DBG Model already loaded in memory: ggml-model-whisper-medium-q5_0.bin
5:08AM DBG GRPC(ggml-model-whisper-medium-q5_0.bin-127.0.0.1:43907): stderr whisper_full_with_state: auto-detected language: ru (p = 0.592074)
5:08AM INF Success ip=127.0.0.1 latency="24.747µs" method=GET status=200 url=/readyz
5:08AM DBG Trascribed: &{Segments:[{Id:0 Start:0s End:5s Text:Белый дождь Tokens:[50364 5697 1414 4851 1070 20037 678 50614]} {Id:1 Start:5s End:10s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:2 Start:10s End:15s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:3 Start:15s End:20s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:4 Start:20s End:25s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:5 Start:25s End:30s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:6 Start:30s End:35s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:7 Start:35s End:40s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:8 Start:40s End:45s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:9 Start:45s End:50s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:10 Start:50s End:55s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:11 Start:55s End:1m0s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:12 Start:1m0s End:1m5s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:13 Start:1m5s End:1m10s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:14 Start:1m10s End:1m15s Text:Долгое время Tokens:[50364 3401 1227 4567 387 12039 50614]} {Id:15 Start:1m15s End:1m17s Text:*музыка* Tokens:[50364 1853 11147 23787 2833 9 50464]}] Text:Белый дождьДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое времяДолгое время*музыка*}
5:08AM INF Success ip=192.168.101.20 latency=19.777624466s method=POST status=200 url=/v1/audio/transcriptions

I tried setting the yaml file and modified it as follows, but it still didn't work.

backend: whisper
name: whisper-medium-q5_0
parameters:
  model: ggml-model-whisper-medium-q5_0.bin
  language: "Chinese"

How can I configuration to select different language?

Finally, I hope to convert a mixed Chinese and English audio file

[Nyralei]

In curl request
-F language="zh"

[Nyralei]

Maybe language: "zh" in yaml would work too

[aef5748]

Thanks for your reply.
To set language is not work in yaml.
The curl can workable, but response content is Simplified Chinese not Traditional Chinese.
I tried to set --initial_prompt via curl, but it is not working in localai.

[Nyralei]

Yes, I think only language, translate and CPU threads are supported in localai. But Go binding of Whisper.cpp does support initial prompt, so it can be added to localai relatively easily.

[Nyralei]

Anyway, try to use whisper.cpp itself first. There is server implementation in "examples" and it does support API too.

[aef5748]

I have add context.SetInitialPrompt("內容: ") in backend/go/transcript.go but it is not working.
Is it need rebuild after any modify?

[Nyralei]

Yes, you need to rebuild