Improved KO CVVC Phonemizer #1122

EX3exp · 2024-04-30T13:25:19Z

Added additional function in BaseKoreanPhonemizer to handle Non-Hangeul graphemes, like tha, thi, vyeo.
Improved KO CVVC Phonemizer:

Completely supports up-to-dated Korean CVVC reclist.
Supports Korean Phoneme variation as same level as KO CBNN/CV/DIFFS/ENUNU Phonemizers.
Supports Special phonemes(which is purposed to support English Phonemes). CV, VC, VV, CC, Batchim will be generated as same as normal Hangeul phonemes. (for example: "커 rreon 씨" = [keo, eo rr, rreo, eo n, n ss, ssi].)
If singer doesn't have some CV phonemes, It flexibly supports alternative phonemes. (like ka, -ka instead of - ka)
If singer doesn't have some VC or CC phonemes, it flexibly ignores those VC / CC phonemes. (or use alternative phonemes, like a g instead of a gw)
Supports phoneme hints input with square bracket and comma, as implemented in BaseKoreanPhonemizer.
Automatically generates end phonemes(like a R), or can input end phonemes with tail notes, as implemented in BaseKoreanPhonemizer.

Should be merged after Fixed bug in KoreanPhonemizerUtil, KO CBNN Phonemizer #1121

This reverts commit 1d5478a, reversing changes made to 9f91b54.

Update Korean Phonemizers (#1121 + #1122)

* Fixed forgetting to set icons * Added icon settings to the style window * Adjust cache filename * update dependencies * Improved translation of error messages for more flexibility * Fix AggregateException message * error log minor fix * comment out re-adding of default expression * sync strings * Update EnglishVCCVPhonemizer.cs * Adds Canadian Raising vowels to the vvExceptions. * Fix error message on `mel_scale` * Update EnglishVCCVPhonemizer.cs * Adds F to ucvs * move default pen selection logic to key handler * fix openutau crash when dragging an invalid zip file into openutau window * Fallbacks for Canadian Raising * Fix loading dialog in preference window * Add message dialog when failed to search singer * Add Korean G2P * Add DiffSingerKoreanG2PPhonemizer * Add flag marge function * delete lines as resolved in pr stakira#1118 * fix track import * refactored Romanize function * fixed bugs & improved DiffSingerKoreanPhonemizer * Add function to handle EN phonemes * Fixed bug that generates " " instead of "ㅇ" in first consonants * Deleted deprecated function in CV,CBNN * Added function to handle Non-Hangeul graphemes * Fixed bug in CBNN that not generates VC with semivowel * Improved KoreanCVVCStandardPronunciationPhonemizer * fixed typo * fixed typo * Fixed to use alternative VC&CC if needed * Fixed to use alternative VCs if needed * fixed typo * fixed typo * Fix `[ng g]` instances turning into `[ng]` when `CCV` is present * Fix preview failure when lyrics replacement contains characters that cannot be used in regular expressions * Fixed force activation of the main window when an error dialog is shown * Add option to move only the cursor and not scroll back when pausing playback * Update pr-test.yml * Update pr-test.yml * Update pr-test.yml * fix for note start at 0 tick * Fix a bug when importing project into an unsaved project * enhance error message for phoneme not in phoneme list or pitch model not found * Update DiffSingerRenderer.cs * enhance error message when the dsdict.yaml under dspitch or dsvariance isn't found * Reverts Avalonia * Revert "Merge pull request stakira#1121 from EX3exp/update-ko-phonemizer-util-2" This reverts commit c451fc1, reversing changes made to 50dfc45. * Revert "Merge pull request stakira#1122 from EX3exp/update-ko-cvvc-phonemizer" This reverts commit 1d5478a, reversing changes made to 9f91b54. * Auto-cleanup prereleases only * added files * added files * added file * Organising the reset menu * Parent window of error dialog is now determined only when necessary * Fixed error in voice color remapping * Fix MargeExpression * Fix failed to launch when failed to open project * Showing positions that failed to render * minor fix * diffsinger: notes consist of only consonants will take up the whole length of the note * Fix load presamp.ini VCPAD * read track names from midi, write track names into midi * presamp.ini allows blank parameters * Fix Draw Pitch Tool * minor fix * Add Overwrite pitch tool * [JA VCV&VCCV] Fix start glottalstop * handle encrypted archive * First linkage with setparam * Add Japanese monophone G2P + add support to Diffsinger Japanese Phonemizer * Add workaround for dash notes (the G2P doesn't recognize it for some reason) * Add some things I forgot * Add some more stuff I forgot * Add yet even more stuff I forgot * Fixed two identical flags in one phoneme * [JA VCV] Support * あ * [JA CVVC] remove unnecessary codes * [JA VCV] Support * あ * [JA CVVC] remove unnecessary codes * Revert "[JA VCV] Support * あ" This reverts commit b87751c. * Revert "[JA CVVC] remove unnecessary codes" This reverts commit 4d45f96. * Fix how starting consonants are handled * Remove leftover code * Ignore slur lyrics when merging notes with Ctrl+U * fix openutau crash when opening a project with a diffsinger voicebank that isn't installed on this PC * Update LocalizedSort.cs * Fix auto-save * Changed font size to be calculated in order of height * add "sc.npz" to VoicebankFiles.cs * add "sc.npz" to Preferences.cs * fix note timing after tempo change for ZH CVVC * use a better variable name * Comment out .frq in VoicebankPublishIgnores * Adjust VC and starting C to tempo * move MsToTickAt to TimeAxis * Fixed crash when unzipping singer fails * string.empty means not translated * fix * fix not loading english dictionary * Multiple fixes * Add some vowels I forgot * Add "Open singers location" menu * fix openutau crash when g2p lyrics helper failed to provide a suggestion * Show InputGesture on notes context menu * Fix wavtool not working when full path contains 2 byte characters * fix test name * Fix unable to save * Adds a experimental build workflow * Update Strings.pl-PL.axaml Updated Polish translation * remove NAudio midi importer * Show error message to install Visual C++ * fix overwrite pitch tool pitch misplaced * clear vibrato and MOD+ after pitch baking * prepare for sync * move batch edit logic from PianoRollViewModel.cs to PianoRollWindow.axaml.cs * Updated Korean strings * fix not showing error dialog * Update pr-test.yml * add 'Install singer' to singer menu in track header * add comment * Added Shebang to Generated Shell File. Shebang added to the MacOS generated resampler shell file for running resamplers with wine. * fix 'add pitch point' misplaced * Trigger Test * Fix error when using exe wavtool on linux The system for running exe wavtools is to create a temporary batch file, but OpenUtau tries to run this batch file directly, which does not work on Linux. This commit adds a check for the operating system and will create a shell script wrapper that runs the batch file through wine if the OS is Linux. A few other changes also had to be made for this to work, such as using different character encodings that did not include byte order marks, because they cause problems both when running shell scripts and when running batch files through wine. Because the current implementation uses the path style appropriate for the operating system, it was writing linux-style paths to the batch file, which was another cause of issues that this commit fixes. Finally, the wavtool and resampler paths both point to the wrapper created by the user during the install process (see link at bottom), but this is not the correct path to give the batch file because windows batch files cannot run ilnux shell scripts; therefore, this commit includes code to extract the path to the exe file from the sh wrapper. https://github.com/stakira/OpenUtau/wiki/Resamplers-and-Wavtools#linux * fix very long splitted note * Fixed issue with .sh parser when path used backslash space * Fix error when using exe wavtool on linux OpenUtau was trying to run a batch file directly when using an external .exe wavtool, which does not work on Linux. This commit includes several changes needed to make .exe wavtools work on Linux. Rather than running a batch file directly, a temporary .sh file is created which runs the batch file through wine. There are a few other changes that had to be made to accomodate this: - Byte order marks cause problems when executing shell scripts and when executing batch files through wine, so there is code to use character encodings that do not include them when writing to the files, but only on Linux. - The Path class always returns linux-style paths when running on Linux, (as it should), but when writing to a .bat file, the paths need to be Windows-style, so this commit includes code to convert the paths to the proper style. - The batch file needs to know the location of the .exe file for the resampler / wavtool, not the location of the .sh wrapper that the user created while installing it. OpenUtau only knows the location of this wrapper, so there is a method to parse the .sh file to resolve the path to the .exe file. * Fixed issue with .sh parser when path used backslash space * Removed debug lines * Updated zh-CN strings * Update System.Text.Json * 已按照建议更改，不过resampler翻译成重采样器原本就存在，改了会不会突兀？ * Updated zh-CN strings * F0編集のサポートとデフォルトのF0データを変更 * Remove excess source code * Phonemizer modifications for pitch editing support * Add .ustx file association on Windows and MacOS * Fixes Strings.pl-PL.axaml * Update OpenUtau.csproj Fix? * Update Strings.pl-PL.axaml Code review: missing strings, typos and awkward transaltions fixed * Update OpenUtau.csproj * Update OpenUtau.csproj * Update OpenUtau.csproj * Update OpenUtau.nsi * Update OpenUtau.csproj * Fix several bugs on pressing Ctrl+Z in TextBoxes * fix nsis quote escaping * replaces portaudio with a new backend * deletes portaudio * deletes naudio and cleanup project * audio fixes * fix yaml race conditions * revives naudio --------- Co-authored-by: yqzhishen <[email protected]> Co-authored-by: StAkira <[email protected]> Co-authored-by: maiko3tattun <[email protected]> Co-authored-by: AnAndroNerd <[email protected]> Co-authored-by: lennyservant <[email protected]> Co-authored-by: oxygen-dioxide <[email protected]> Co-authored-by: Cardroid <[email protected]> Co-authored-by: EX3 <[email protected]> Co-authored-by: cadlaxa <[email protected]> Co-authored-by: LitMus <[email protected]> Co-authored-by: Maiko <[email protected]> Co-authored-by: Maiko <[email protected]> Co-authored-by: Lotte V <[email protected]> Co-authored-by: tansansuisui <[email protected]> Co-authored-by: Astel123457 <[email protected]> Co-authored-by: KluelessKisa <[email protected]> Co-authored-by: ochakochatblanc <[email protected]> Co-authored-by: parallelepiped2718 <> Co-authored-by: ssmzhn <[email protected]> Co-authored-by: RedBlackAka <[email protected]> Co-authored-by: RedBlackAka <[email protected]>

EX3exp added 5 commits April 30, 2024 21:26

Added function to handle Non-Hangeul graphemes

c33bc3a

Improved KoreanCVVCStandardPronunciationPhonemizer

b0dab2a

Fixed to use alternative VC&CC if needed

89e71e1

fixed typo

6f44c66

fixed typo

75e2921

stakira merged commit 1d5478a into stakira:master May 10, 2024
0 of 3 checks passed

stakira added a commit that referenced this pull request May 10, 2024

Revert "Merge pull request #1122 from EX3exp/update-ko-cvvc-phonemizer"

1d4cf31

This reverts commit 1d5478a, reversing changes made to 9f91b54.

EX3exp mentioned this pull request May 10, 2024

Update Korean Phonemizers (#1121 + #1122) #1132

Merged

stakira added a commit that referenced this pull request Jun 9, 2024

Merge pull request #1132 from EX3exp/ko-cvvc-phonemizer

5ad4e8b

Update Korean Phonemizers (#1121 + #1122)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improved KO CVVC Phonemizer #1122

Improved KO CVVC Phonemizer #1122

EX3exp commented Apr 30, 2024 •

edited

Loading

Improved KO CVVC Phonemizer #1122

Improved KO CVVC Phonemizer #1122

Conversation

EX3exp commented Apr 30, 2024 • edited Loading

EX3exp commented Apr 30, 2024 •

edited

Loading