Localization and Dubbing

Synthetic dubbing must pass timing, stress and pronunciation, then consent

Related

Synchronization LicenseTimbre

The problem

One film needs five or six language versions. Synthetic voices already sound close, but the lip sync gives them away. A Chinese line of eight characters becomes twelve syllables in English while the mouth on screen is still moving. Names, brands and acronyms are also read wrongly, and the audience hears it in the first second.

How it works

Dubbing is bound by three constraints, the length of the original line, the stress that matches the actor's gesture, and pronunciation that fits local usage. A synthetic voice is generated from text length, not picture length, so it either rushes or overruns. Prosody follows punctuation and will not pause because a character raises a hand. Voice cloning also raises consent questions, where the training data came from, whether commercial use is allowed, and whether it may be trained again.

Practice

Dubbing is not a by-product of translation and needs its own schedule. Adapt the translation for length first, marking the character or syllable count of each line and rewriting whatever overruns instead of speeding it up. Give the engine a pronunciation list that spells out every name, brand and acronym. Where lip sync must be tight, use phoneme alignment to reorder the sentence, or accept a voice-over. The contract should define the scope and term of the voice licence and whether training is permitted, and the delivery should state which passages are synthetic. Have a native speaker audition the result rather than trusting your own ear.

What to listen for

In Mahler's Wenn mein Schatz Hochzeit macht, how each German syllable sits on a note; in the Dies irae of Mozart's Requiem, how the choir aligns its consonants; in the Bulgarian folk song Katerino Mome, the vowels and ornaments of a language that is not English; in Holmes's Aubade-Habanera, French liaison and stress. These are the details a synthetic voice drops first.

Sources

• 配音时长适配与合成语音的授权问题，依本地化行业通识核对，未抄原文
• Dubbing timing and synthetic voice consent, checked against general localization practice, not copied

Used for fact-checking only; all prose is original.