Synthetic Vocals

Synthetic singing handles demos, backing and languages you cannot record; long notes and emotion still give it away.

Related

TimbreVoiceVibrato

The problem

Synthetic singing means a model performs the melody and the words, not that your own recording is shifted. It holds pitch and pronounces text well enough for three jobs: a demo someone else will sing, a backing layer under a real voice, and a language you cannot cast. It is a poor choice for the lead on a release unless listeners know the voice is synthetic.

How it works

The system separates pitch, duration and phonemes. Lyrics become phonemes, the melody supplies pitch and length, and timbre comes from recorded samples or a trained acoustic model. Separate control makes changing words, key or tempo cheap. It also means someone must decide how the line is sung: where to breathe, which syllable to weight, how a long note ends. If nobody decides, the model returns an average of those choices.

Practice

Decide the use first. A demo can be fully synthetic. On a release, replace the lead with a singer and keep the synthetic part as backing. Leave room for breaths when writing the melody, keep phrases under eight to ten syllables, and keep the range inside an octave and a half. Fade long notes in the MIDI and add a very low breath layer. Have a native speaker check stress and liaison in a foreign language. In the mix, narrow the synthetic voice slightly and clear space around 2 to 5 kHz for the real voice or dialogue.

What to listen for

In Du bist die Ruh, listen to where the long phrases break for breath. In Erlkönig, listen to how one singer colours three characters. In Monteverdi's madrigal, listen to how each line moves on its own. In the hymn, listen to how voices blend with the organ. Synthetic singing fails in exactly these places: breath, character, independence and blend.

Sources

• 合成人声的工作方式，依公开研究摘要与产品文档核对，未抄原文
• How synthetic singing is built, checked against public research summaries, no text copied

Used for fact-checking only; all prose is original.