Websocket Data For Genders and Races

[arcsidian]

Hello,

I started a Unity project that utilizes your websocket option to get all the dialogues into unity in order to manipulate it, get it voiced by elevenlabs and do a few other things to it. And so far it's been going well. But I realized that I can't find out the gender or the race of the speaker directly.

What I get from it is:

string Speaker;
string Type;
string Payload;
object Voice {
    int Id
    string Name
    float Rate
    float Volume
    string VoiceName
    int EnabledBackend
}
string Source;

Is there a way to get the gender and the race of the speaker? or at least the gender? or do I have to do it the painful way by building a huge database inside my program identifying every name.
I'm hoping that I'm missing something here because I've only really been paying attention to Speaker and Payload.

[karashiiro]

There is not, but gender and race should just be a couple dozen options, not something worth a database for. I think that can fit in the voice name, unless I'm missing something.

[arcsidian]

Thanks for the quick reply. I meant building a database that has the names of all NPCs, and each time a new message is sent through the websocket, I'd check the speaker in the database to find out its information (race, gender..etc)

[karashiiro]

Ah, I see — gender is in the voice name, at least, but I don't have anything to get race, yet. I've been inconsistent in the past about where gender info is as well, since I haven't been getting any feedback on this and don't know what's most convenient to work against. Previously, gender was a separate field, and there was no voice preset info, but I ended up changing that to avoid clients having to implement any game-specific functionality. I do think the plugin should be responsible for exposing that information if needed, but I'm not really sure how, exactly.

Anyways, I haven't put as much thought into this as I'd like, so suggestions and/or PRs are welcome for a more usable protocol.

[arcsidian]

I had no idea Voice.VoiceName had the gender, that is incredibly helpful, thank you very much. I remember once printing the contents of Voice and all the results were numbers, including VoiceName, so I thought they were IDs saved for something specific. I'll pay more attention to VoiceName.

The way I've been thinking about it is that I'm trying to make at least one or two voices for each gender and each race in ElevenLabs for variety, that's why I originally needed both gender and race, but just getting the gender saves me a lot of work. I do hope that in the future we can get more data going into the websocket, it will help a lot considering the rate of growth we are witnessing with AI voices and the custom options they offer. Increasing the parameters broadcasted in WebsocketBackend.cs:43 will be beneficial.