Reading a Model Card
Read the limitations and training data before the scores
Related
The most useful part of a card
Two model cards: one says it ranks first on an overall benchmark, the other says long Chinese sentences drift and structure collapses past three minutes. The second one helps you decide. A model card earns its keep by naming the inputs that break, not by winning a table.
Four fields, four follow-up questions
| Field | Usual wording | What to ask |
|---|---|---|
| Training data | Public audio and licensed catalogues | Which languages, eras and recording types; is there an opt-out route |
| Limitations | May contain artefacts | Which ensemble, what length, which vocal style fails |
| Evaluation | Subjective score of 4.2 | Who rated, how many, was it blind, against what reference |
| Licence | Commercial use, credit required | What is said separately about weights, outputs and training |
The training-data row is usually dispatched with "millions of tracks". Ask about composition instead of count: does it include the languages and traditions you need, is it mostly recordings or MIDI, how much of it is voice. The limitations row has the same problem. "May contain artefacts" says nothing; "instrumental pieces over two minutes start repeating" is useful.
Three kinds of evaluation numbers
The same claim of beating the previous version can come from an internal listening panel, a crowd preference test or a blind AB. An internal panel is small, a crowd test measures first impressions, and blind AB only says which of two clips someone preferred. None transfers directly to your task. Find the evaluation set first: which material, what length, does it cover your language and ensemble. If that is missing, read the score as advertising.
Where the model card stops
"Not evaluated" is a prompt, not a defect. Build a small benchmark from five to ten of your own pieces, with fixed prompts, fixed seeds and a fixed listener, and write down how each one fails. A model card is a filter, not an acceptance test. It removes clearly unsuitable models; the rest has to be tried on your own material. Listen after reading the card, not before, or your ears will start arguing for the score.
Listen in the library
All tracks from midicn-lib — inline badge is the license tier (C1 commercial / C2 non-commercial / C3 study only).
Along the same line
Sources
- Mitchell et al., Model Cards for Model Reporting（2019）与 Gebru et al., Datasheets for Datasets（2018）。
- 生成式音频主观评测方法，参考公开研究综述。
- 模型许可与用途限制，来自官方条款。