Is it really useful to document Android voices?

[HadrienGardeur]

Given the state of voice selection on Android, I'm really wondering if documenting them is helpful. Quick reminder of the current issues with Android voice selection:

• Chrome on Android returns a list of languages instead of a list of voices #12
• Chrome on Android returns unavailable languages in its voice list #14
• Chrome on Android defaults to another language without a default voice read-aloud-best-practices#6

To make things worse, these voice names are also localized, which means that the more we extend our language support in this project, the more we need to document additional altNames.

The irony of it all is that we can actually document these Android voices better in Chrome OS than in Android itself:

web-speech-recommended-voices/json/en.json

Lines 1243 to 1259 in 1d24762

	{
	"label": "Female voice 1 (US)",
	"name": "Android Speech Recognition and Synthesis from Google en-us-x-tpc-network",
	"note": "This pre-loaded voice on ChromeOS is tied to the Android compatibility layer and requires network access for this version, which is very much a requirement for using Chrome OS anyway. Fallbacks to natural voice and offline variant as well.",
	"altNames": [
	"Google US English 5 (Natural)",
	"Android Speech Recognition and Synthesis from Google en-us-x-tpc-local"
	],
	"language": "en-US",
	"gender": "female",
	"rate": 1,
	"pitch": 1,
	"os": [
	"ChromeOS"
	],
	"preloaded": true
	},

I'm at a point where I'm wondering if I should give up on documenting these voices, as it really feels pointless.

If I had to summarize a list of pros/cons for having them in the list of recommended voices:

Pros:

• Slightly better voice name than what Android provides
• Implementers can rely strictly on recommended voices to offer a decent selector on all platforms

Cons

• Due to the bugs that I've documented, most of these voices won't work anyway
• Additional work whenever we add another language, for very little in return

I'm tempted to keep the status quo until we turn this repo into code. With code we could either:

• handle these altNames variations programmatically with locales
• and/or return other voices where there is no match for recommended voices

Any thoughts from anyone else on this? cc @chocolatkey @mickael-menu @danielweck

[Falcosc]

Thanks for all the work! You are amazing.

I prefer code as well, then you don't have to deal with handling how to enroll breaking interface changes if the old structure does not work anymore.

A function which returns voices in your recommended order would be amazing.
Currently, I concat recommendations of 2 of your language lists (user locale + English) to order the voices without removing any of them.

Result order looks like this:

1. User language voices in recommended order
2. English voices in recommend order
3. all other language and not detected voices

To give you some feedback, how I use your data: I reduce your structure to flat like this:

name
altName
altName
altName
name
altName
altName
altName

And then I order the available voices based on the index of name match in the flat list.

There is one disadvantage for this approach: high quality Apple voices are maybe not on top if the first installed high quality voice is for example "Lee (Premium)" since Lee is the last apple voice in your list.

[Falcosc]

I guess it will not fix this one:
https://communities.apple.com/de/thread/254066932?answerId=257918442622&sortBy=best#257918442622

It's just too much effort to work on iOS with all these issues.

I will revisit iOS in 1-2 Years when the EU law did give us enough developers complaining about the apple implementation of TTS. Every week I discover a new iOS problem.

Thank you

[Falcosc]

Microsoft is not much better, feels like a competition who has the most obvious bugs ;)
Multilingual Voices from Edge stop processing after reading "&" for all German multilingual voices.
It gets correctly read, but it throws an exception after that.
I don't know what's worse, reading Homograph between the words and inability to understand phrases between german words or having TTS skip the rest of the text on & 😅

Luckily, the other edge voices are still better than iOS and Firefox. It is even possible to barely understand English words without multilingual support. On other browsers, it's often impossible if you don't have the text as a clue of how to interpret the mess.
So yes, for German, Edge is the only solution so far if there is some internet slang involved.

[HadrienGardeur]

I will revisit iOS in 1-2 Years when the EU law did give us enough developers complaining about the apple implementation of TTS. Every week I discover a new iOS problem.

One of my goal with this project is to document these issues. There are many bugs and limitations that get in the way of implementing a decent read aloud experience, but they're not centralized somewhere.

I've started a Github project specifically for that at: https://github.com/users/HadrienGardeur/projects/2

If you encounter any bug, feel free to either open an issue on this repo or the main one dedicated to the project (I'll triage and tag them accordingly). You can also reach out directly by email if that's better for you.

I'll make sure that several organizations working on accessibility are made aware of them, and hopefully this will give us enough leverage to prioritize some of these bugs at Apple/Google/Microsoft/Mozilla.

Multilingual Voices from Edge stop processing after reading "&" for all German multilingual voices.

Yikes. Hopefully they can solve that issue on the server side, these multilingual voices are brand new and might be more prone to errors than others.

This is a good example of the kind of bugs that need to be files in the project dedicated to known-issues.

[Falcosc]

I have created 3 issues to go into more details. My other iOS issues are not related to Speech. But the "you need a Mac for web inspection" was the last straw before I asked by self "why supporting at all?" if the experience sucks anyway for them because of the voice quality.

[HadrienGardeur]

Thanks! I'll triage these issues tomorrow so don't be surprised if I move and/or rename them.

[Falcosc]

One more reason to find a more generic solution with fuzzy detection for Android Voices is the differences between iOS and Android.

On Android, you can have JavaScript Running while the display is off. This is interesting for any complex, long-running use case of Text to Speech.
On iOS you need an App to do this. Maybe Apple don't want to work on adding an exception for special JS declarations because they don't like if people are able to bypass the need of the App store by making browsers too powerful. For example audio streaming is a big market where apple wants to have a share, so quite convenient that streaming providers can not create iOS web apps for there platform ;)

On the other hand, users of long-running applications are more likely to spend more time with trying out voices and don't need recommendations as badly as website visitors.

[HadrienGardeur]

On the other hand, users of long-running applications are more likely to spend more time with trying out voices and don't need recommendations as badly as website visitors.

Well the bad news is that Android makes this nearly impossible due to its current approach for voice selection (see issues listed in my initial message). It's better if you're building a native app, but not that great either since the API doesn't return a first name or a gender for voices.

On Android, you can have JavaScript Running while the display is off. This is interesting for any complex, long-running use case of Text to Speech.
On iOS you need an App to do this.

Apple doesn't like having anything running in the background on iOS/iPadOS, even with native apps. You usually have to work extra hard to do this and/or use specialized APIs.

[Falcosc]

Cool, Edge on Android is even more broken. I told you it's a challenge between Apple and Microsoft who has the most broken Browser TTS. I thought the Edge bug in your tracker is something new or only for a specific thing, no does not work at all since a long time.

Can you remind me what year we have? Maybe we should skip TTS and directly generate a Video based on the Text prompt and use that to hear the text, that seems to be easier than TTS in browser ;)

[Falcosc]

Edge Android uses the Edge Windows strings.

Since Edge Android issues are bugs while Chrome issues feel more like features I feel we get a solution on Edge first if at all.
Chrome voices are not worth the hassle, I would skip then until they get competitive. Maybe it makes more sense to have a look on other android browsers with much less market share firstwhile waitung for the Edge fix.
Meanwhile I use the Edge get voice list workarround. Thanks for your work.

[HadrienGardeur]

Now that we're truly implementing this work beyond a demo, I think that I can provide an answer to the initial question: "Is it really useful to document Android voices?".

And the answer is… not really. I'll continue to document one localized name (in the target language) and to indicate with localizedName that these voices are localized using Android's pattern for voices: language Region where language can be downcased or capitalized depending on capitalization rules for that language.

Going beyond that seems unnecessary because:

• these values are not truly voices (see relevant issue)
• localization can be handled using Intl.DisplayNames
• and worst-case scenario they will be displayed as-is if we don't filter out non-recommended languages

I'm closing the discussion for now.