Using Edge-TTS with Websocket for free "neural" voices

[v8bt97yn]

I couldn't find any information on how to do this, so hopefully this helps somebody. Please feel free to improve it if you have suggestions.

This is a decent free alternative to using the system voices. The script uses Edge-TTS to assign a random voice and pitch to every NPC you speak to. The voice + pitch is recorded in JSON so that each NPC retains their assigned voice when you speak with them (based on Name/ID/Gender). The script uses an array of voice options, so you can change them to anything you want. You can also modify the JSON to assign any voice you wish to the NPC's you've already spoken to.

---

Setup:

1. Download the zip using the link below
2. Extract the zip wherever you want, the audio and json will be stored in the ffxiv-tts directory
3. Install and Enable TextToTalk In-Game
4. Copy the Websocket IP and Port from the screenshot below
5. Once your Websocket is started, run the main.py from the ffxiv-tts directory
6. Done!

---

Websocket Settings:

---

Sample Video:
https://github.com/user-attachments/assets/8ab7bcb9-2659-42fe-a04c-55e2f1c4cf3e

---

Download:
ffxiv-tts-20240829.zip

---

Resources:
Edge-TTS
Edge-TTS Voice List
How To: Verify Websocket Is Running
How To: Configure Python On Windows
How To: Add More Voices
How To: Autostart With XIVLauncher

[saitoh183]

Seems awesome, could you explain how to set it up? do i just need to run main.py and then setup the websocket like your screenshot? @Tyler-Altgrave

[v8bt97yn]

@saitoh183 no real setup needed, you just need to have Websocket enabled in-game with the same IP and Port as the screenshot before running the script. Extract the download wherever you want it, install the requirements and run the main.py

[saitoh183]

so i opened a terminal and ran pip install -r requirements.txt
this installed the requirements then rans main.py which is gives a blinking cursor...so i assume it is started.

In game i copied your screenshot and restarted the game ...still nothing... sorry king of a noob when it comes to websockets...

[v8bt97yn]

You'll need to make sure the websocket in-game shows "Started on ws://127.0.0.1:12746/Messages"

When everything is running properly, you'll see the following logs on start,

You'll also see new logs when speaking to NPC's for the first time

[v8bt97yn]

You can also download Postman and set up a connection to the WebSocket if you want to verify its running
If the Websocket connects, you'll see messages being logged in the Postman app

[saitoh183]

i got it..!!! i just had to run python main.py...i was running just ./main.py and it was opening visual studio code and then had nothing... Thanks

[saitoh183]

i wish i could speed up the vocals like when you use system...it is just a tad slow... but on the flip side, the male voice sounds so much better than default windows robotic sound...

[v8bt97yn]

Glad you got it running!

I uploaded a fix for the pitch changes a few minutes ago.
If you haven't downloaded that, you'll need to change the line pitch = random.randint(-10, 1) to pitch = random.randint(-10, -1) in the main.py. The code only handles negative values, so it'll eventually crash if you don't update that line. Just save and run the main.py again after that and everything should be good to go

If you've already run into the error, open the speaker_details.json and find the pitch that has a value of 1 and change it to -1

[saitoh183]

quick question, where are the voices taken from? can we add more?

[saitoh183]

quick question, where are the voices taken from? can we add more?

I think i figured it out...just checked edge-tts git

[saitoh183]

@Tyler-Altgrave Thank you for this. It has made the TTS experience so much more enjoyable. When i encounter a voice i find doesnt fit, i can just replace it in the json file ...its great!

[v8bt97yn]

I'm glad you're enjoying it, I just started playing the MSQ again and I'm loving not having to read 😁

…

On Sun, Aug 18, 2024, 6:07 PM saitoh183 ***@***.***> wrote: @Tyler-Altgrave <https://github.com/Tyler-Altgrave> Thank you for this. It has made the TTS experience so much more enjoyable. When i encounter a voice i find doesnt fit, i can just replace it in the json file ...its great! — Reply to this email directly, view it on GitHub <#226 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWWGMEYJ4XGB4P6NZ2WQF4DZSELLLAVCNFSM6AAAAABMWQEOMKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMZXGYZDCOI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[saitoh183]

@Tyler-Altgrave can i ask for a modification? Sometimes characters come back in other scenarios and are assigned a new ID because of it
example Meffrid in Quarymill

"name": "Meffrid",
        "id": 1008112,
        
"name": "Meffrid",
        "id": 1000332,

So we end up with different voices. I fixed it by editing my speaker_details.json. I dont know if there is a elegant way to handle this. I saw that your if statement needs ID and Name to be the same for it declaire the entry already exist but seems other scenarios change some NPCs ID.

[v8bt97yn]

I tried to make it more specific so that you wouldn't always get the same voice for every guard in town, but the game seems to send back different IDs for the same NPCs. Currently it checks for Name/ID/Gender, but you can update the check to only include the Name.

[saitoh183]

I tried to make it more specific so that you wouldn't always get the same voice for every guard in town, but the game seems to send back different IDs for the same NPCs. Currently it checks for Name/ID/Gender, but you can update the check to only include the Name.

I think a nested If statement might work... first check for the ID if it is false then check the name. If it is false, create a new entry,

[saitoh183]

I tried to make it more specific so that you wouldn't always get the same voice for every guard in town, but the game seems to send back different IDs for the same NPCs. Currently it checks for Name/ID/Gender, but you can update the check to only include the Name.

@Tyler-Altgrave i have refined the script, Please try it out and give it a try

Summary of Changes:
ID Update Handling:

• Added a check to overwrite the id if the speaker's name is found but the ID is different. This will ensure that the character keeps the same voice if found in a new scenario with a different ID.

Announcements:

• The script now prints a message when a new speaker entry is created and when an existing speaker's ID is updated.

[saitoh183]

Configure Python On Windows

Alright so for those that are like me and had a bit of trouble figuring it out.

• Download and extract this into your desired location

• Download and install python 3.x for windows (Python 3)

• If you install Python for all users using custom make sure you check add python to Path also you will need to add C:\Users\<username>\AppData\Roaming\Python\Python312\Scripts to your Path under system
  

• Open the folder you extracted the app to and right click in it and choose Open in Terminal.
  

• In the terminal type in pip install -r requirements.txt

• Restart the terminal by opening it again.

• in terminal type python .\main.py

[saitoh183]

How to add voices to the script

open main.py in notepad++ or Visual Studio code and navigate to line 13 and 14

open a terminal window and type in edge-tts.exe --list >> "path to save file" (ie: edge-tts.exe --list >>d:\ttslist.txt) This will save all the possible voices that can be used to a file called ttslist.txt

Now navigate to your saved file and open it.

Navigate to https://ttsfree.com/text-to-speech#google_vignette . This site will allow you to preview all the voices in the list so you can see which one you want to add to the file. You want to listen to the voices under TTS Server 2.

Once you find a voice, simply search for it in your file. Once you find it copy the Name and paste it in the main.py at the end of the Array of the proper gender. Look at my screenshot above for reference. Make sure you do not add a comma to the last entry in the array but there must be a comma in between entries.

Once you are done, save main.py and relaunch main.py

[saitoh183]

Create auto launch in XIVLauncher

Here is how you can setup ffxiv-tts to Auto start.
Just follow theses screenshots:



-d d:\ffxiv-tts powershell -Command "python main.py"

If you want to run the script in debug mode, you can add -debug after main.py

NOTE

If you use OTP, you just need to hit cancel to get back to the main screen.

[Benjo0815]

Hello, first of all thanks for creating the script.
I have the problem that every character has a male voice.
On the console I always get none for gender. Here is an example:
New speaker entry created: Feuermondreigen-Werberin with ID 1051480 and gender None

i run your script on an Arch Linux system with Python version 3.12.4.
FFXIV runs on my Steam Deck (SteamOS) via XIVLauncher.

In your script I have replaced the English voices with German voices and changed the IP to that of my Steam Deck.

Everything seems to run normally, except for the male voices, even for female characters.

What could be the reason for this?
I would appreciate any help :)

[v8bt97yn]

You might want to check the Edge-TTS or TextToTalk issues to see if theres a known issue with Linux.

If you haven't already, you should first check that the websocket is delivering the gender. It would be in json under voice.name example

I'm honestly not really familiar with how python interacts with other operating systems. I've only used the script on my Windows laptop. Its super basic and just passes off whats sent from the websocket to edge-tts, so there shouldn't be any reason it wouldn't work universally

[saitoh183]

@Benjo0815 post the lines you modified

[Benjo0815]

Thanks for the quick replies!

@Tyler-Altgrave I have checked it in Postman and the gender is indeed not delivered. I think the problem is rather Text to Talk Plugin in connection with linux.

@saitoh183
I have changed this lines
male_voices = ["en-US-ChristopherNeural", "en-GB-RyanNeural", "en-AU-WilliamNeural", "en-US-EricNeural", "en-US-GuyNeural", "en-US-RogerNeural", "en-US-SteffanNeural", "en-KE-ChilembaNeural", "en-IE-ConnorNeural", "en-GB-ThomasNeural"]
female_voices = ["en-US-AriaNeural", "en-GB-SoniaNeural", "en-AU-NatashaNeural", "en-US-JennyNeural", "en-US-MichelleNeural", "en-NZ-MollyNeural", "en-KE-AsiliaNeural", "en-IE-EmilyNeural", "en-GB-LibbyNeural", "en-GB-MaisieNeural"]

to this lines
male_voices = ["de-DE-ConradNeural", "de-DE-KillianNeural", "de-AT-JonasNeural", "de-CH-JanNeural"]
female_voices = ["de-DE-AmalaNeural", "de-DE-KatjaNeural", "de-AT-IngridNeural", "de-CH-LeniNeural"]

and this line
uri = "ws://127.0.0.1:12746/Messages"

to this line
uri = "ws://192.168.1.228:12746/Messages"

[saitoh183]

@Tyler-Altgrave
New version > ffxiv-tts20240823.1.zip (when you added it to the original post, i will remove it from here)

User Configuration Handling:

• I have defined ws_uri, male_list, female_list at the top for easy user configuration.
• The ws_uri, male_voices, and female_voices variables now default to predefined values if the user configuration is empty.

Debug parameter added:

• The script now includes the -debug flag for enabling or disabling debug mode. When run with debug, it will print additional debugging information. (ex: python ./main.py -debug)
• It will print all the data retrieved from the websocket

WebSocket Connection Retry Logic: (Replace the sleep at start of script)

• Connect_websocket function that attempts to connect to the WebSocket and retries every 10 seconds if the connection fails for a maximum of 5 attempts before quitting the script.
• The listen function now uses connect_websocket to establish and maintain the WebSocket connection.
• If using the auto start with XIV Launcher, the script will quit by itself after the 5 attempts once you have exited the game.

Automatic Reconnection:

• If the WebSocket connection is closed unexpectedly, the script will attempt to reconnect automatically.

@Benjo0815

Try this version with the debug flag and post the result. You will have to put your configs back but i made a section for that at the start of the script so that you should not need to scroll through the script to replace values.

[Benjo0815]

@saitoh183

I have now tried your version of the script. With the debug flag I have the following output during a conversation:

Received data: {'Type': 'Say', 'Speaker': 'Vianne', 'NpcId': 1001309, 'Race': 'Elezen', 'BodyType': 'Adult', 'Payload': 'Ich überlege, mir mein Zepter etwas aufmöbeln zu lassen. Die Goldschmiede hier gravieren ausgezeichnete Ornamente, da würden meinen Gefährten die Ohren schlackern. Entsprechend gesalzen sind die Preise allerdings auch.', 'PayloadTemplate': 'Ich überlege, mir mein Zepter etwas aufmöbeln zu lassen. Die Goldschmiede hier gravieren ausgezeichnete Ornamente, da würden meinen Gefährten die Ohren schlackern. Entsprechend gesalzen sind die Preise allerdings auch.', 'Voice': {'Id': -1, 'Name': 'None', 'Rate': 0, 'Volume': 0, 'VoiceName': None, 'EnabledBackend': 1}, 'StuttersRemoved': True, 'Source': 'AddonTalk', 'ChatType': None, 'Language': 'German'}
Extracted gender: None
New speaker entry created: Vianne with ID 1001309 and gender None

I think that the problem is more related to the Text-To-Talk plugin than to this script.

Thank you very much for your effort!