DiffSinger

Open-source singing synthesis built on your own voicebanks and phoneme labels.

Model profile

Vendor

DiffSinger 社区

Form

本地模型

Input

歌词 · MIDI · 参考音频

Output

人声轨 · 音频

Licence

代码许可与声库、训练数据许可分开核对，自建声库需演唱者同意

Verified

2026-10-30

Related

VoicePitchVibrato

What it lets you finish

Feed it lyrics, a MIDI melody and phoneme labels and get a sung vocal. You can use an existing community voicebank or train a private one from your own recordings. It suits fully offline work, long-term reuse of one timbre, and custom projects for a particular language or delivery. It returns a vocal track; arrangement and mixing stay with you.

Inputs and outputs

Input is lyrics, MIDI and note lengths; training also needs recorded data with phoneme labels. Output is a vocal track, exportable dry. The whole pipeline runs locally, and you manage model files and voicebanks yourself. Language coverage depends on the phoneme dictionary and the voicebank, and not every language has a ready route. The listening examples and instances below are references for this kind of output, not this model's generations.

Strengths

• Fully local, so material and voicebanks stay on your machine, which suits long projects and confidential content.
• Pitch and duration come straight from MIDI, so rhythm and intonation can be exact rather than guessed from a prompt.
• You can train a voicebank on your own recordings for a timbre nobody else has.
• Open source, so the model and front end can be modified and wired into your own batch pipeline.

Weaknesses and failure modes

• The entry cost is high: environment, dependencies, voicebank and phoneme dictionary are yours to set up, and a first successful run often takes a full day. A reproducible test is a clean install from scratch, noting where it stalls.
• Mandarin polyphones, erhua and neutral tones depend on dictionary rules, and misreadings need manual phoneme edits; one wrong character sends you back into the line.
• Voicebank quality varies enormously, and community banks differ in recording and labelling, so swapping banks feels like swapping systems.
• Expression and breath are hand-drawn, and default output is flat, especially on long notes.
• Training needs clean recordings and correct labels; with too little data, timbre drifts and syllables drop.
• It produces voice only, so reverb, compression and sibilance work happen in the DAW.

Terms, price and alternatives

Confirm: code licence and voicebank licence are separate, and each community bank carries its own terms on commercial use; training on a real singer needs that singer's consent, and data provenance must be explainable. The software is open source, with cost in compute and time. Alternatives: ACE Studio for something that works out of the box with support; MusicGen when you need instruments and no voice. This suits teams that need offline operation, a private timbre, and the ability to maintain it themselves.

Sources

• 项目结构、训练流程与输入形式，依公开仓库说明核对，未抄原文
• 失败模式依公开用户报告与自建声库经验归纳，未引用具体评测数据

Used for fact-checking only; all prose is original.