I want to produce video content that uses a computer-generated voice so as to keep me anonymous (and speak in a somewhat more "consistent" manner than I do, and generate a "clean" audio that's inherently static-free), but still with high-quality speech synthesis that sounds natural. I've found that even the best AI systems don't always seem to understand what I have in mind with a transcript of the text - and really, in many cases it would be impossible to guess: I often want to emphasize words that aren't the most natural choice, speak with an unusual cadence or rhythm, etc. I also find that sentence-level intonation is a generally hard problem (even for humans!) and I'm generally not satisfied with the results of ordinary text-to-speech.

My idea, therefore, is to record myself reading the text, and then use some kind of software that can synthesize a voice speaking the same words - with a consistent, pre-set timbre and standard pronunciation (i.e. obscuring my local accent and personal speech characteristics), but emulating my intonation, rhythm, stress, pitch variation etc. Since it would necessarily detect my voice and then generate output that is only speech, this would also implicitly remove background noise.

An ideal solution:

  • is open-source, or at least gratis

  • works on Linux

Bonus points for:

  • ability to customize the extent to which it tries to emulate my voice vs. the "standard" voice

  • ability to transcribe the text internally (it's no problem to provide a script as separate input, since I'd be reading from one anyway; but I'd like to be able to double-check what it's "hearing")

  • ability to parameterize the output speech in other ways - in particular, to make it speak faster or slower than my recording

  • ability to "sing" recognizably in addition to speaking

Not required or valued:

  • a nice UI (I'm happy to use a library and write some code around it)

  • real-time processing (since this is for video production - in fact, I'd greatly prefer to be able to throw extra computing time at the problem)

  • a single all-in-one solution (it's fine if I need to connect a speech-to-marked-up-text system to the reverse, as long as the "marked up text" can actually represent the kinds of details described above and I don't have to spend hours trying to edit that markup by hand)

  • Not sure I understand. Do you mean just a voice changer?
    – Destroy666
    Commented Apr 28 at 14:10
  • Rather than modifying the input voice, I want it to re-synthesize so as to match certain characteristics, but "sound like" a specific stock voice that's already provided. So if the "stock" voice sounds like Microsoft Sam, everyone who used the software would get an output that sounds "like Microsoft Sam, but following the user's cues" for stress, pacing etc. It would, as I imagine it, work by analyzing the input voice for those features, and then using them to parameterize its usual synthesis. Commented Apr 28 at 19:20

1 Answer 1



Please have a look at OpenVoice. While I have not tested it myself, it seems to match your requirements.

OpenVoice has been powering the instant voice cloning capability of myshell.ai since May 2023. Until Nov 2023, the voice cloning model has been used tens of millions of times by users worldwide, and witnessed the explosive user growth on the platform.

Voice cloning seems to be what you are looking for, right? Checkout the video snippet in the README.md: https://github.com/myshell-ai/OpenVoice

Docker containers are available. It's even free for commercial use.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.