You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jan 18, 2024. It is now read-only.
i want to train a model to perform code-switching TTS / voice conversion for only 2 languages: Vietnamese and English. i assume the model should perform well with training data of 1 speaker in Vietnamese and 1 in English, each with a decent number of utterances (~15hrs). my reason is that since there are only 2 speakers & a lot of data, the model should be able to learn the speaker embedding for each, even by remembering (overfitting). similarly for the language-dependent encoder. but I see some of your comments saying it's better to include other languages, each with multiple speakers in the training data, even if you don't use it in inference. why?
The text was updated successfully, but these errors were encountered:
Hi @thivux , I can't speak for @Tomiinek and only remember this vaguely, but I believe the problem / reason you want to have more languages AND more speakers is that you need to model to dissociate the languages & speakers from each other, to be able to generalize and code-switch. If there's only 1 speaker per language, the speaker embeddings are too closely tied to the language and the model won't be very good at switching to the other language – you'll see how your training goes. I believe you shouldn't need more languages, but having more speakers in each of the two languages, even with less or lower-quality data, should help.
hi there, thank you for the interesting work!
i want to train a model to perform code-switching TTS / voice conversion for only 2 languages: Vietnamese and English. i assume the model should perform well with training data of 1 speaker in Vietnamese and 1 in English, each with a decent number of utterances (~15hrs). my reason is that since there are only 2 speakers & a lot of data, the model should be able to learn the speaker embedding for each, even by remembering (overfitting). similarly for the language-dependent encoder. but I see some of your comments saying it's better to include other languages, each with multiple speakers in the training data, even if you don't use it in inference. why?
The text was updated successfully, but these errors were encountered: