Guided synthesis - API Improvement

[Patchethium]

内容

Following #252, as I'm digging down into the GUI part I found some parts of the API design are obviously not clever enough and improved them.

Now the AudioQuery has a new optional section guidedInfo at root, containing all the information needed for guided synthesis and would directly be passed to the engine.

{
    "accent_phrases": [
       "..."
    ],
    "guidedInfo": {
        "enabled": true,
        "audioPath": "/home/.../sample.wav",
        "normalize": true,
        "precise": true
    }
}

As shown above, I replaced the uploaded file with a full path to the file in string so I can get rid of the form data and use a simpler design. As a result, guided_synthesis and guided_accent_phrases now are exactly the same with the synthesis API, which I think would ease the GUI development a lot.

More details could be found in the change of README.

PS: Considering the usage in GUI, guided_accent_phrases doesn't actually work like accent_phrases. That's the reason why I removed most of the parameters as well as the text and is_kana.

[github-actions]

Coverage Result

Resultを開く

Name	Stmts	Miss	Cover
voicevox_engine/init.py	1	0
voicevox_engine/acoustic_feature_extractor.py	75	0
voicevox_engine/dev/synthesis_engine/init.py	2	0
voicevox_engine/dev/synthesis_engine/mock.py	40	4
voicevox_engine/experimental/init.py	0	0
voicevox_engine/experimental/guided_extractor.py	128	94
voicevox_engine/experimental/julius4seg/init.py	0	0
voicevox_engine/experimental/julius4seg/converter.py	298	295
voicevox_engine/experimental/julius4seg/sp_inserter.py	116	89
voicevox_engine/full_context_label.py	162	3
voicevox_engine/kana_parser.py	86	1
voicevox_engine/model.py	163	7
voicevox_engine/mora_list.py	4	0
voicevox_engine/part_of_speech_data.py	5	0
voicevox_engine/preset/Preset.py	12	0
voicevox_engine/preset/PresetLoader.py	34	1
voicevox_engine/preset/init.py	3	0
voicevox_engine/synthesis_engine/init.py	5	0
voicevox_engine/synthesis_engine/core_wrapper.py	167	132
voicevox_engine/synthesis_engine/make_synthesis_engines.py	52	43
voicevox_engine/synthesis_engine/synthesis_engine.py	181	69
voicevox_engine/synthesis_engine/synthesis_engine_base.py	68	9
voicevox_engine/user_dict.py	98	10
voicevox_engine/utility/init.py	3	0
voicevox_engine/utility/connect_base64_waves.py	35	3
voicevox_engine/utility/engine_root.py	9	2
TOTAL	1747	762

[Hiroshiba]

I noticed from reading the README you wrote, guided synthesis is at the frame level!
This has one positive and two negative consequences.

The advantage is of course the higher resolution of the input. It is undefined behavior for the model, but the user will feel happy.

The first disadvantage is that the VOICEVOX UI (mora level) does not allow adjustment of the pitch or length. Users will have to re-create the audio for the guide.

The second disadvantage is that it may become unavailable in the future. In fact, I am currently developing a decoder model for high quality, which does not allow frame-level F0 input.

I think it would be ok to either keep the frame level or change to mora level, since we are in the experimental stage right now. But we have to choose one of them....
Sorry for the late notice...

If we change to the mora level, the code would be very straightforward because we would create an API to get the AccentPhrase from the voice for the guide.

---

書いてくれたREADMEを読んで気づきました。guided synthesisはフレームレベルですね！
これには１つの嬉しいことと、２つの損があります。

利点はもちろん入力の解像度が高いことです。モデルにとっては未定義動作ですが、ユーザーは嬉しく感じるでしょう。

１つ目の欠点は、VOICEVOXのUI（モーラレベル）で音高や長さを微調整できない点です。ユーザーはガイド用の音声を作り直す必要があります。

２つ目の欠点は、将来的に利用不可になるかもしれない点です。実はいま高品質用デコーダーモデルを作成中なのですが、これはフレームレベルのF0入力ができません。

フレームレベルのままにするのか、モーラレベルに変更するか、今は実験段階なのでどちらでも良いと思います。が、どちらかを選ぶ必要があります･･･。
気づくのが遅れてしまって申し訳ないです･･･。

もしモーラレベルに変更すると、ガイド用音声からAccentPhraseを得るAPIを作ることでコードがとてもわかりやすくなりそうです。

[Patchethium]

In fact, I am currently developing a decoder model for high quality, which does not allow frame-level F0 input.

Interesting, I wonder how you'd handle the alignment.

Since I've already completed the GUI part I don't feel like giving up on a working feature. I suggest we keep this feature until the new architecture is introduced into this repository. No matter how it turns out to be there'll be a breaking change where we can remove this API by the way.

[Hiroshiba]

Ok, I understand!

[Hiroshiba]

Sorry for the wait!

[Patchethium]

Should be okay now.

[Patchethium]

Another week is passing, how's it going on now?

[Hiroshiba]

LGTM！！

Sorry to keep you waiting!

[Patchethium]

Great, you may also want to check out the pr on GUI side so we can finish this feature for the next release.