Skyrim modders have a new machine learning tool that turns text to realistic NPC speech

The modding community has a new tool to help them create original dialog for their mods. The app is called VASynth and is available on the NexusMods website. VASynth uses AI to convert text into realistic synthesized speech.
VASynth creator Dan Ruta used NPC voices from various Bethesda video games to train the tool’s machine learning AI models. He published multiple flavors of the app, including variations for Skyrim, Fallout 4, Oblivion, Fallout New Vegas, Morrowind, and Fallout 3. Each has a selection of models to download optionally.

Ruta also plans to release a Starfield version “soon.” He presumably means sometime after Bethesda launches the game, which still doesn’t have a release date. In the meantime, the modder will continue adding additional models and work on training existing voices to improve quality.

Ruta posted a demo (below) of how VASynth works, which is narrated by several voice models. The demo is relatively crude, and the speech often sounds unnatural. However, the characters explain and provide examples of how to improve the spoken words using punctuation, spelling tricks, pitch, and speed controls. Ultimately, users can record realistic voices with a little effort.

Another modder posted a Skyrim trailer (top) as an example of what kind of results are possible with VASynth. While some phrases don’t sound quite right, many others are very convincing. Fearing that some might mistake the voices created with VASynth as the actual Bethesda voice actors, Ruta warns users not to misuse the app.

“To keep things fair, avoid using the tool in an offensive/explicit manner,” he cautions in the notes of each version. “Make it obvious where you can in descriptions that the voice samples are generated, and not from the original voice actors. Any issues you cause with this are on you.”

One of the first examples of a gameplay mod using VASynth is called “Positive Undressed Reactions.” As then name suggests, the mod causes Skyrim NPCs to complement players if they walk around nude. It’s a rather silly application of the tech, but one can easily see this tool being used to create scripted quest expansions for any of the supported Bethesda games.