Humaneness Voice: which checkpoint works best?

A paired, public listening comparison of the SFT4 starting model, the weighted SFT5 continuation, its rank-128 RL adapter, and the new one-exposure SFT6 full fine-tune. Small and Large are compared separately. Every column receives the same literal text, performance prompt, seed and reference crop.

At a glance

3519 / 3519 full-suite predictions currently available. Missing columns say “pending”; they are never replaced with another model's audio. The page updates as checkpoints finish evaluation. Green SFT6 columns labelled FINAL are the completed runs: Small step 1,372 and Large step 1,661. A separate development-selected SFT6 column can be an earlier checkpoint; its step is shown explicitly. Final does not mean best. Every SFT6 column uses exactly the same 207 prompts, seeds and reference crops as the earlier models.

Earlier 1.1 Alpha grid · MP3s, exact prompts and complete score annotations.

Primary priorities: Content Enjoyment and requested-emotion realization. Speaker identity, intelligibility, instruction following and genuineness are reported alongside them, not concealed when they worsen.

What each training run means

SFT4 baseline: the weights before the ElevenLabs/Gaston multilingual continuation. Large is SFT4 step 288. Small uses its explicitly documented Real + Gemini starting weights; the historical SFT number is a convenient comparison label, not an assertion that architectures or step counters match.
SFT5: the previous weighted full fine-tune: original Gemini once, multilingual Gemini twice, Gaston five times and ElevenLabs six times. It starts from SFT4, with one warmup/cosine schedule.
RL: the final rank-128 adapter on SFT5's final model, not a full fine-tune. Eight generated candidates form a relative-reward group. The objective is VESPO sequence-level soft policy optimization with GDPO-style component-wise reward normalization, an experimental GRPO-family adaptation without a learned critic. Sequence-importance weighting limits unstable changes between the rollout policy and the updated policy; it is not the flow-matching GROW method used by the separate Voice Converter. See the VESPO paper and GDPO paper. Half the training groups include a distinct 5–15-second reference, half have only timbre/performance instructions. Relative reward combines normalized frozen judges: with reference, 40% Content Enjoyment + 30% VoiceCLAP performance-text similarity + 30% Orange speaker identity; without reference, 60% CE + 40% VoiceCLAP. Candidates failing the transcript-preservation gate do not train. Rewards are standardized within each group; gradients update the adapter while the base stays frozen. This can improve one judge without improving all other properties.
SFT6: a fresh full fine-tune directly from SFT4, not from SFT5 or RL. Every eligible physical target is presented once after deduplication and global shuffle, including the added French-accent material. It has a new warmup/cosine schedule. Real-speech mining runs separately and is not mixed into this SFT6. Large has 212,577 targets / 1,782.28 hours across all five languages. Small has 87,796 targets / 730.67 hours: Gemini English/German only; ElevenLabs includes English/German/French; French-accent material is retained. Each source has repetition factor one. Both use approximately 50% distinct-reference and 50% learned-name conditioning.

New: completed Alpha standard RL-v2 adapters

The additional purple RL-v2 columns are new final rank-128 adapters on the same frozen Alpha/SFT5 parent, not the SFT6-based experiments. Small completed2,616 attempted G8 groups with2,256 optimizer updates; Large completed2,530 groups with2,344 updates. Rejected paired groups received no update and remain recorded.

RL-v2 adds separate calibrated emotion and talking-style matching and genuineness: with a reference,20%Content Enjoyment +20%genuineness +25%emotion matching +25%style matching +10%identity. Without a reference, identity is absent and the remaining components are renormalized. Emotion and style target intensities come from the source target audio, not the reference voice. Each active component is group-normalized before VESPO/GDPO updates. The original RL-v1 columns remain independently labelled with their earlier recipe.

New RL-v2 audio uses exactly the same207 texts, literal prompts, seeds, reference crops and FP32/temperature0.7 inference contract as the matched controls. Alias-only cases are retained even though RL-v2 training uses no aliases in its no-reference arm. Training-judge reuse and known prompts mean these measurements are developer evidence, not a new independent benchmark.

All six new RL runs: final production checkpoints

The two Alpha standard RL-v2 columns and four SFT6 RL columns are independently trained rank-128 adapters. Every new column uses the same207 prompts, seeds and reference crops, and the same inference and scoring settings. Purple SFT6 RL players must be compared with their green FINAL SFT6 parent of the same size, not with a different starting model.

Small SFT6 standard:2,616 attemptedG8 groups /2,324 updates; Small SFT6 emotion-focused:2,616 /2,384. Standard uses20%CE,20%genuineness,25%emotion,25%style,10%identity with reference. Emotion-focused uses20%CE,40%emotion,30%style,10%identity; the10%identity remainder is the documented interpretation of the requested90%partial recipe. No-reference groups omit identity and renormalize the other weights.

Large SFT6 standard:2,530 attemptedgroups /2,360 updates; Large SFT6 emotion-focused:2,530 /2,378. These are the repaired, completed production runs, NOT the earlier failed smokes. Large adds exactly10%bounded inverse-WER reward,1/(1+WER), and reduces the other weights proportionally. Standard reference weights:18%CE,18%genuineness,22.5%emotion,22.5%style,9%identity,10%WER. Emotion-focused reference weights:18%CE,36%emotion,27%style,9%identity,10%WER. Without reference, WER remains10% and the other90% is renormalized without identity. Large peakLR is3.75e-6 versus Small1e-5; both have5%warmup/cosine.

All six production runs are complete. A pending player means its matched prediction or scoring is still being backfilled, not that training is still running. Rejected training groups remain logged; they are not optimizer updates. Every saved final adapter also retains its optimizer and per-rank RNG states.

How to read the measurements

Content Enjoyment (CE), production quality, genuineness, burst blending and requested-emotion strength are frozen Humaneness Ears Medium predictions. These are model-estimated scores, not a human listening survey. “Genuineness” here is the Ears estimate, not a fresh invocation of the historical standalone genuineness classifier.

VoiceCLAP instructions compares the prediction audio with only GENERAL and round SCRIPT delivery directions. Spoken words and square-bracket timing markers are removed from the text embedding. VoiceCLAP reference is audio-to-audio performance similarity and is not an identity measure. Speaker identity is Orange WavLM-ID 250-dimensional cosine; timbre is a different Orange 128-dimensional metric. For name-only cases the audition is used only for scoring, never as a model input. Neither metric proves identity. Gaston and ElevenLabs names were introduced after SFT4; their name-only SFT4 cases include previously unseen aliases and should not be interpreted as a pure voice-cloning comparison.

WER is corpus word error rate from Parakeet v3. Lower is better. Mean ± SD describes variation across cases, not confidence intervals. Pure no-reference cases have no hidden voice name and no speaker-similarity score. Comparison groups are shown separately below.

All audio is scored from the same 128-kbps mono MP3 representation. Every arm uses FP32 arithmetic on the BF16-rounded model, TF32 disabled, audio temperature 0.7, top-p 0.95 and top-k 50. No best-of-N, seed cherry-picking or failed-take filtering is used. Long outputs beyond Ears' 30-second window are flagged in the annotations.

Full-suite results

Reference supplied

Model	Scored cases	Corpus WER ↓	Content Enjoyment ↑ (mean ± sample SD)	Requested emotion ↑ (mean ± sample SD)	VoiceCLAP instructions ↑ (mean ± sample SD)	Genuineness ↑ (mean ± sample SD)	Speaker identity ↑ (mean ± sample SD)	Timbre ↑ (mean ± sample SD)	Production quality ↑ (mean ± sample SD)	Vocal-burst blend ↑ (mean ± sample SD)
Large SFT4 starting baseline	121	7.626%	5.821 ± 0.467	1.456 ± 0.957	0.014 ± 0.120	4.307 ± 0.277	0.403 ± 0.166	0.519 ± 0.165	7.952 ± 0.200	8.012 ± 0.499
Large SFT5 · step 3,055 · exact RL parent	121	4.812%	5.988 ± 0.361	1.673 ± 1.062	0.034 ± 0.110	4.373 ± 0.208	0.416 ± 0.180	0.521 ± 0.180	7.949 ± 0.184	8.144 ± 0.468
Large Alpha RL-v1 (earlier recipe) · 2,370 updates	121	2.678%	6.336 ± 0.273	1.810 ± 1.043	0.022 ± 0.108	4.453 ± 0.243	0.414 ± 0.177	0.517 ± 0.175	8.016 ± 0.147	8.041 ± 0.490
Large Alpha standard RL-v2 · FINAL · 2,344 updates	121	2.224%	6.276 ± 0.262	1.689 ± 0.977	0.021 ± 0.123	4.558 ± 0.233	0.417 ± 0.159	0.510 ± 0.166	7.973 ± 0.143	8.131 ± 0.475
Large SFT6 · step 1,660 · development-selected	121	14.072%	6.011 ± 0.404	1.597 ± 1.030	0.029 ± 0.112	4.363 ± 0.224	0.415 ± 0.166	0.515 ± 0.166	7.979 ± 0.149	8.119 ± 0.514
Large SFT6 · step 1,661 · FINAL	121	4.085%	6.037 ± 0.372	1.693 ± 1.065	0.030 ± 0.115	4.395 ± 0.231	0.414 ± 0.182	0.521 ± 0.177	7.976 ± 0.149	8.115 ± 0.513
Large SFT6 standard RL + 10% inverse-WER · FINAL · 2,360 updates	121	1.498%	6.309 ± 0.237	1.661 ± 0.982	0.018 ± 0.109	4.552 ± 0.246	0.446 ± 0.163	0.554 ± 0.161	8.009 ± 0.117	8.066 ± 0.497
Large SFT6 emotion RL + 10% inverse-WER · FINAL · 2,378 updates	121	3.223%	6.263 ± 0.341	1.689 ± 1.036	0.031 ± 0.120	4.359 ± 0.246	0.432 ± 0.181	0.527 ± 0.173	8.039 ± 0.160	7.976 ± 0.527
Small SFT4 starting baseline	121	1.997%	6.146 ± 0.352	1.675 ± 0.957	0.017 ± 0.125	4.321 ± 0.252	0.365 ± 0.160	0.447 ± 0.162	8.049 ± 0.172	7.908 ± 0.491
Small SFT5 · step 2,442	121	6.355%	6.058 ± 0.376	1.609 ± 0.999	0.009 ± 0.126	4.350 ± 0.255	0.377 ± 0.166	0.481 ± 0.196	8.002 ± 0.165	7.944 ± 0.454
Small SFT5 · step 6,109 · exact RL parent	121	5.856%	6.033 ± 0.361	1.595 ± 1.118	0.003 ± 0.127	4.351 ± 0.250	0.411 ± 0.168	0.515 ± 0.172	7.976 ± 0.159	7.996 ± 0.485
Small Alpha RL-v1 (earlier recipe) · 2,434 updates	121	2.133%	6.410 ± 0.167	1.813 ± 1.053	0.009 ± 0.126	4.466 ± 0.248	0.414 ± 0.161	0.506 ± 0.177	8.046 ± 0.105	7.934 ± 0.457
Small Alpha standard RL-v2 · FINAL · 2,256 updates	121	1.997%	6.391 ± 0.155	1.616 ± 1.053	-0.007 ± 0.114	4.612 ± 0.240	0.408 ± 0.168	0.503 ± 0.182	7.957 ± 0.112	7.934 ± 0.389
Small SFT6 · step 548 · development-selected	121	2.905%	6.121 ± 0.317	1.818 ± 1.096	0.017 ± 0.124	4.330 ± 0.282	0.389 ± 0.162	0.482 ± 0.165	8.014 ± 0.150	7.998 ± 0.533
Small SFT6 · step 1,372 · FINAL	121	3.995%	6.137 ± 0.396	1.636 ± 1.023	0.013 ± 0.121	4.356 ± 0.258	0.398 ± 0.161	0.487 ± 0.175	8.047 ± 0.140	7.982 ± 0.506
Small SFT6 standard RL · FINAL · 2,324 updates	121	0.999%	6.474 ± 0.150	1.548 ± 0.996	-0.013 ± 0.123	4.669 ± 0.186	0.402 ± 0.148	0.485 ± 0.185	7.998 ± 0.106	7.954 ± 0.366
Small SFT6 emotion RL · FINAL · 2,384 updates	121	1.770%	6.405 ± 0.368	1.672 ± 1.018	0.002 ± 0.131	4.445 ± 0.234	0.399 ± 0.159	0.485 ± 0.175	8.060 ± 0.183	7.923 ± 0.474

Learned speaker name only

Model	Scored cases	Corpus WER ↓	Content Enjoyment ↑ (mean ± sample SD)	Requested emotion ↑ (mean ± sample SD)	VoiceCLAP instructions ↑ (mean ± sample SD)	Genuineness ↑ (mean ± sample SD)	Speaker identity ↑ (mean ± sample SD)	Timbre ↑ (mean ± sample SD)	Production quality ↑ (mean ± sample SD)	Vocal-burst blend ↑ (mean ± sample SD)
Large SFT4 starting baseline	51	12.837%	5.645 ± 0.536	1.255 ± 0.836	-0.071 ± 0.103	4.424 ± 0.261	0.256 ± 0.118	0.123 ± 0.173	7.859 ± 0.210	8.097 ± 0.547
Large SFT5 · step 3,055 · exact RL parent	51	5.178%	5.934 ± 0.494	1.807 ± 0.978	-0.065 ± 0.118	4.454 ± 0.293	0.269 ± 0.136	0.124 ± 0.186	7.953 ± 0.182	8.122 ± 0.506
Large Alpha RL-v1 (earlier recipe) · 2,370 updates	51	2.805%	6.355 ± 0.265	1.777 ± 0.915	-0.096 ± 0.133	4.576 ± 0.249	0.250 ± 0.144	0.119 ± 0.176	7.980 ± 0.127	8.211 ± 0.412
Large Alpha standard RL-v2 · FINAL · 2,344 updates	51	2.481%	6.298 ± 0.243	1.588 ± 0.947	-0.100 ± 0.125	4.683 ± 0.186	0.257 ± 0.130	0.112 ± 0.179	7.952 ± 0.093	8.258 ± 0.452
Large SFT6 · step 1,660 · development-selected	51	3.776%	5.998 ± 0.308	1.897 ± 0.850	-0.059 ± 0.122	4.481 ± 0.247	0.250 ± 0.127	0.115 ± 0.198	7.983 ± 0.107	8.201 ± 0.491
Large SFT6 · step 1,661 · FINAL	51	7.983%	5.905 ± 0.452	1.801 ± 0.983	-0.075 ± 0.120	4.476 ± 0.221	0.248 ± 0.117	0.117 ± 0.191	7.952 ± 0.173	8.105 ± 0.507
Large SFT6 standard RL + 10% inverse-WER · FINAL · 2,360 updates	51	1.402%	6.312 ± 0.296	1.796 ± 0.883	-0.074 ± 0.143	4.632 ± 0.205	0.257 ± 0.135	0.118 ± 0.187	7.964 ± 0.132	8.173 ± 0.396
Large SFT6 emotion RL + 10% inverse-WER · FINAL · 2,378 updates	51	1.079%	6.349 ± 0.128	1.806 ± 0.944	-0.084 ± 0.130	4.445 ± 0.263	0.233 ± 0.116	0.111 ± 0.177	8.036 ± 0.076	8.094 ± 0.541
Small SFT4 starting baseline	51	6.472%	5.981 ± 0.626	1.588 ± 0.892	-0.088 ± 0.131	4.226 ± 0.282	0.217 ± 0.136	0.124 ± 0.190	7.916 ± 0.298	7.785 ± 0.606
Small SFT5 · step 2,442	51	4.207%	6.091 ± 0.252	1.817 ± 0.917	-0.087 ± 0.145	4.426 ± 0.233	0.302 ± 0.160	0.157 ± 0.249	7.986 ± 0.097	8.069 ± 0.429
Small SFT5 · step 6,109 · exact RL parent	51	5.070%	6.019 ± 0.313	1.842 ± 0.990	-0.061 ± 0.142	4.368 ± 0.260	0.301 ± 0.156	0.160 ± 0.246	7.936 ± 0.136	8.115 ± 0.479
Small Alpha RL-v1 (earlier recipe) · 2,434 updates	51	6.041%	6.411 ± 0.142	1.744 ± 0.977	-0.109 ± 0.134	4.562 ± 0.228	0.299 ± 0.161	0.148 ± 0.233	8.039 ± 0.094	8.204 ± 0.430
Small Alpha standard RL-v2 · FINAL · 2,256 updates	51	3.560%	6.392 ± 0.133	1.530 ± 0.939	-0.118 ± 0.144	4.732 ± 0.216	0.307 ± 0.141	0.174 ± 0.230	7.953 ± 0.097	8.165 ± 0.332
Small SFT6 · step 548 · development-selected	51	12.406%	5.753 ± 0.891	1.776 ± 0.972	-0.064 ± 0.135	4.318 ± 0.278	0.233 ± 0.154	0.099 ± 0.187	7.882 ± 0.304	8.055 ± 0.568
Small SFT6 · step 1,372 · FINAL	51	10.248%	5.951 ± 0.503	1.512 ± 1.027	-0.081 ± 0.145	4.378 ± 0.239	0.264 ± 0.137	0.131 ± 0.217	7.935 ± 0.220	8.136 ± 0.531
Small SFT6 standard RL · FINAL · 2,324 updates	51	3.236%	6.372 ± 0.345	1.598 ± 0.818	-0.110 ± 0.146	4.610 ± 0.283	0.258 ± 0.166	0.123 ± 0.220	7.959 ± 0.165	8.058 ± 0.523
Small SFT6 emotion RL · FINAL · 2,384 updates	51	5.717%	6.234 ± 0.671	1.613 ± 0.943	-0.104 ± 0.148	4.409 ± 0.299	0.273 ± 0.156	0.171 ± 0.221	7.937 ± 0.266	7.981 ± 0.505

Neither reference nor name

Model	Scored cases	Corpus WER ↓	Content Enjoyment ↑ (mean ± sample SD)	Requested emotion ↑ (mean ± sample SD)	VoiceCLAP instructions ↑ (mean ± sample SD)	Genuineness ↑ (mean ± sample SD)	Speaker identity ↑ (mean ± sample SD)	Timbre ↑ (mean ± sample SD)	Production quality ↑ (mean ± sample SD)	Vocal-burst blend ↑ (mean ± sample SD)
Large SFT4 starting baseline	35	7.705%	5.851 ± 0.429	1.543 ± 1.045	0.039 ± 0.128	4.348 ± 0.305	N/A	N/A	7.967 ± 0.161	8.097 ± 0.524
Large SFT5 · step 3,055 · exact RL parent	35	4.262%	5.895 ± 0.387	1.847 ± 1.080	0.038 ± 0.121	4.369 ± 0.270	N/A	N/A	7.855 ± 0.283	8.147 ± 0.432
Large Alpha RL-v1 (earlier recipe) · 2,370 updates	35	0.164%	6.379 ± 0.174	2.037 ± 0.953	0.018 ± 0.108	4.499 ± 0.256	N/A	N/A	7.986 ± 0.143	8.075 ± 0.502
Large Alpha standard RL-v2 · FINAL · 2,344 updates	35	0.820%	6.338 ± 0.197	1.813 ± 0.883	-0.018 ± 0.116	4.598 ± 0.254	N/A	N/A	7.916 ± 0.094	8.147 ± 0.543
Large SFT6 · step 1,660 · development-selected	35	3.443%	6.020 ± 0.307	1.820 ± 1.053	0.033 ± 0.104	4.388 ± 0.323	N/A	N/A	7.984 ± 0.144	8.111 ± 0.436
Large SFT6 · step 1,661 · FINAL	35	1.967%	6.017 ± 0.286	1.768 ± 1.052	0.030 ± 0.125	4.351 ± 0.307	N/A	N/A	8.002 ± 0.118	8.140 ± 0.534
Large SFT6 standard RL + 10% inverse-WER · FINAL · 2,360 updates	35	0.656%	6.332 ± 0.175	1.748 ± 0.935	-0.004 ± 0.119	4.619 ± 0.230	N/A	N/A	7.974 ± 0.153	8.190 ± 0.481
Large SFT6 emotion RL + 10% inverse-WER · FINAL · 2,378 updates	35	0.328%	6.290 ± 0.176	1.862 ± 0.864	0.004 ± 0.136	4.437 ± 0.250	N/A	N/A	8.034 ± 0.176	8.081 ± 0.499
Small SFT4 starting baseline	35	6.721%	6.051 ± 0.432	1.383 ± 0.835	0.048 ± 0.128	4.321 ± 0.257	N/A	N/A	8.028 ± 0.198	7.954 ± 0.545
Small SFT5 · step 2,442	35	5.410%	6.097 ± 0.323	1.766 ± 1.070	0.051 ± 0.128	4.360 ± 0.272	N/A	N/A	7.992 ± 0.130	8.087 ± 0.470
Small SFT5 · step 6,109 · exact RL parent	35	2.459%	6.069 ± 0.264	1.898 ± 1.087	0.062 ± 0.129	4.286 ± 0.315	N/A	N/A	7.999 ± 0.132	8.042 ± 0.423
Small Alpha RL-v1 (earlier recipe) · 2,434 updates	35	2.295%	6.434 ± 0.162	1.910 ± 1.042	0.036 ± 0.112	4.445 ± 0.242	N/A	N/A	8.026 ± 0.113	8.065 ± 0.389
Small Alpha standard RL-v2 · FINAL · 2,256 updates	35	0.492%	6.449 ± 0.162	1.711 ± 1.014	0.042 ± 0.105	4.651 ± 0.209	N/A	N/A	7.929 ± 0.111	7.986 ± 0.414
Small SFT6 · step 548 · development-selected	35	1.148%	6.168 ± 0.193	1.693 ± 0.976	0.081 ± 0.119	4.338 ± 0.313	N/A	N/A	8.032 ± 0.110	7.998 ± 0.456
Small SFT6 · step 1,372 · FINAL	35	5.574%	6.064 ± 0.344	1.624 ± 0.917	0.061 ± 0.106	4.314 ± 0.287	N/A	N/A	8.006 ± 0.209	8.021 ± 0.518
Small SFT6 standard RL · FINAL · 2,324 updates	35	0.000%	6.479 ± 0.135	1.504 ± 0.895	0.016 ± 0.122	4.651 ± 0.199	N/A	N/A	7.945 ± 0.096	8.110 ± 0.422
Small SFT6 emotion RL · FINAL · 2,384 updates	35	0.656%	6.448 ± 0.148	1.722 ± 0.931	0.043 ± 0.144	4.414 ± 0.215	N/A	N/A	8.037 ± 0.091	8.035 ± 0.448

What the completed comparisons show

Small RL-v1 versus its exact SFT5 parent, reference inputs (121 cases): CE +0.377; requested emotion +0.218; VoiceCLAP instructions +0.006; speaker identity +0.003; genuineness +0.115; WER -3.72 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small RL-v1 versus its exact SFT5 parent, alias inputs (51 cases): CE +0.392; requested emotion -0.098; VoiceCLAP instructions -0.048; speaker identity -0.002; genuineness +0.194; WER +0.97 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small RL-v1 versus its exact SFT5 parent, none inputs (35 cases): CE +0.365; requested emotion +0.012; VoiceCLAP instructions -0.026; genuineness +0.159; WER -0.16 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Alpha standard RL-v2 versus its exact SFT5 parent, reference inputs (121 cases): CE +0.358; requested emotion +0.021; VoiceCLAP instructions -0.010; speaker identity -0.004; genuineness +0.261; WER -3.86 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Alpha standard RL-v2 versus its exact SFT5 parent, alias inputs (51 cases): CE +0.372; requested emotion -0.312; VoiceCLAP instructions -0.058; speaker identity +0.006; genuineness +0.364; WER -1.51 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Alpha standard RL-v2 versus its exact SFT5 parent, none inputs (35 cases): CE +0.380; requested emotion -0.188; VoiceCLAP instructions -0.020; genuineness +0.365; WER -1.97 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Small SFT6 standard RL · FINAL · 2,324 updates versus its exact SFT6 parent, reference inputs (121 cases): CE +0.337; requested emotion -0.087; VoiceCLAP instructions -0.027; speaker identity +0.004; genuineness +0.313; WER -3.00 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Small SFT6 standard RL · FINAL · 2,324 updates versus its exact SFT6 parent, alias inputs (51 cases): CE +0.421; requested emotion +0.086; VoiceCLAP instructions -0.029; speaker identity -0.006; genuineness +0.232; WER -7.01 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Small SFT6 standard RL · FINAL · 2,324 updates versus its exact SFT6 parent, none inputs (35 cases): CE +0.415; requested emotion -0.120; VoiceCLAP instructions -0.045; genuineness +0.338; WER -5.57 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Small SFT6 emotion RL · FINAL · 2,384 updates versus its exact SFT6 parent, reference inputs (121 cases): CE +0.268; requested emotion +0.036; VoiceCLAP instructions -0.011; speaker identity +0.001; genuineness +0.089; WER -2.22 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Small SFT6 emotion RL · FINAL · 2,384 updates versus its exact SFT6 parent, alias inputs (51 cases): CE +0.283; requested emotion +0.100; VoiceCLAP instructions -0.023; speaker identity +0.010; genuineness +0.031; WER -4.53 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Small Small SFT6 emotion RL · FINAL · 2,384 updates versus its exact SFT6 parent, none inputs (35 cases): CE +0.385; requested emotion +0.099; VoiceCLAP instructions -0.018; genuineness +0.101; WER -4.92 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large RL-v1 versus its exact SFT5 parent, reference inputs (121 cases): CE +0.348; requested emotion +0.137; VoiceCLAP instructions -0.012; speaker identity -0.002; genuineness +0.080; WER -2.13 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large RL-v1 versus its exact SFT5 parent, alias inputs (51 cases): CE +0.422; requested emotion -0.030; VoiceCLAP instructions -0.031; speaker identity -0.019; genuineness +0.123; WER -2.37 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large RL-v1 versus its exact SFT5 parent, none inputs (35 cases): CE +0.483; requested emotion +0.190; VoiceCLAP instructions -0.020; genuineness +0.131; WER -4.10 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Alpha standard RL-v2 versus its exact SFT5 parent, reference inputs (121 cases): CE +0.288; requested emotion +0.016; VoiceCLAP instructions -0.013; speaker identity +0.002; genuineness +0.185; WER -2.59 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Alpha standard RL-v2 versus its exact SFT5 parent, alias inputs (51 cases): CE +0.364; requested emotion -0.219; VoiceCLAP instructions -0.035; speaker identity -0.012; genuineness +0.230; WER -2.70 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Alpha standard RL-v2 versus its exact SFT5 parent, none inputs (35 cases): CE +0.442; requested emotion -0.034; VoiceCLAP instructions -0.056; genuineness +0.229; WER -3.44 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Large SFT6 standard RL + 10% inverse-WER · FINAL · 2,360 updates versus its exact SFT6 parent, reference inputs (121 cases): CE +0.273; requested emotion -0.032; VoiceCLAP instructions -0.011; speaker identity +0.032; genuineness +0.157; WER -2.59 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Large SFT6 standard RL + 10% inverse-WER · FINAL · 2,360 updates versus its exact SFT6 parent, alias inputs (51 cases): CE +0.407; requested emotion -0.005; VoiceCLAP instructions +0.001; speaker identity +0.008; genuineness +0.156; WER -6.58 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Large SFT6 standard RL + 10% inverse-WER · FINAL · 2,360 updates versus its exact SFT6 parent, none inputs (35 cases): CE +0.315; requested emotion -0.021; VoiceCLAP instructions -0.034; genuineness +0.268; WER -1.31 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Large SFT6 emotion RL + 10% inverse-WER · FINAL · 2,378 updates versus its exact SFT6 parent, reference inputs (121 cases): CE +0.226; requested emotion -0.004; VoiceCLAP instructions +0.001; speaker identity +0.018; genuineness -0.036; WER -0.86 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Large SFT6 emotion RL + 10% inverse-WER · FINAL · 2,378 updates versus its exact SFT6 parent, alias inputs (51 cases): CE +0.444; requested emotion +0.005; VoiceCLAP instructions -0.009; speaker identity -0.016; genuineness -0.031; WER -6.90 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.
Large Large SFT6 emotion RL + 10% inverse-WER · FINAL · 2,378 updates versus its exact SFT6 parent, none inputs (35 cases): CE +0.273; requested emotion +0.094; VoiceCLAP instructions -0.027; genuineness +0.086; WER -1.64 percentage points. These are paired fixed-case aggregate changes, not a universal quality guarantee.

No overall winner is declared while SFT4/SFT6 or checkpoint screening is incomplete. A gain in CE does not imply improved identity or emotion realization; inspect the directions of the separate metrics and listen to the same case.

Checkpoint screening and selection

Each complete SFT5/SFT6 snapshot receives a fixed 48-case development panel: all 40 emotion labels plus eight genuine no-reference cases. The historical 40-emotion panel mixes English/reference and German/name-only inputs; these factors are confounded. Within-case checkpoint comparisons remain paired; do not infer a causal reference benefit from those groups.

Selection equally weights standardized paired changes in CE and requested-emotion strength relative to SFT4. WER must not exceed SFT4 by more than one absolute percentage point. If no checkpoint passes, the page explicitly labels the trade-off. The selected models then receive all 207 listening cases. Selection is provisional while more snapshots are pending. This is a developer comparison on known prompts, not an unbiased held-out benchmark. Training-speaker/reference overlap and judge reuse can inflate apparent gains.

Checkpoint	Cases	WER ↓	CE ↑	Requested emotion ↑	VoiceCLAP ↑	Genuineness ↑	Dev selection score	WER guard
Small SFT4 starting baseline	48	2.18%	6.139	1.655	-0.071	4.321	baseline / fixed RL	—
Small SFT5 · step 1,221	48	2.86%	6.064	1.765	-0.086	4.427	-0.103	PASS
Small SFT5 · step 2,442 — selected	48	1.77%	6.146	1.761	-0.093	4.422	0.060	PASS
Small SFT5 · step 2,569	48	3.00%	6.115	1.769	-0.087	4.458	0.001	PASS
Small SFT5 · step 3,663	48	4.90%	6.065	1.746	-0.078	4.373	-0.107	TRADE-OFF
Small SFT5 · step 3,775	48	3.00%	6.071	1.781	-0.073	4.407	-0.082	PASS
Small SFT5 · step 4,884	48	5.18%	6.050	1.732	-0.081	4.382	-0.145	TRADE-OFF
Small SFT5 · step 6,031	48	4.63%	6.038	1.779	-0.079	4.398	-0.147	TRADE-OFF
Small SFT5 · step 6,105	48	3.95%	6.051	1.693	-0.084	4.412	-0.159	TRADE-OFF
Small SFT5 · step 6,109	48	1.63%	6.125	1.814	-0.086	4.404	0.041	PASS
Small SFT6 · step 274	48	4.50%	6.119	1.593	-0.088	4.322	-0.066	TRADE-OFF
Small SFT6 · step 548 — selected	48	2.32%	6.089	1.747	-0.075	4.375	-0.061	PASS
Small SFT6 · step 822	48	9.40%	5.955	1.660	-0.087	4.362	-0.365	TRADE-OFF
Small SFT6 · step 1,096	48	9.95%	5.972	1.489	-0.098	4.396	-0.403	TRADE-OFF
Small SFT6 · step 1,370	48	8.04%	5.981	1.525	-0.103	4.361	-0.371	TRADE-OFF
Small SFT6 · step 1,372	48	6.27%	6.055	1.603	-0.080	4.373	-0.190	TRADE-OFF
Small Alpha RL-v1 (earlier recipe) · 2,434 updates	48	6.27%	6.404	1.915	-0.071	4.521	baseline / fixed RL	—
Small Alpha standard RL-v2 · FINAL · 2,256 updates	48	4.63%	6.422	1.750	-0.091	4.703	baseline / fixed RL	—
Large SFT4 starting baseline	48	9.95%	5.694	1.402	-0.085	4.429	baseline / fixed RL	—
Large SFT5 · step 611	48	1.63%	5.989	1.718	-0.071	4.518	0.472	PASS
Large SFT5 · step 1,222	48	7.49%	5.908	1.714	-0.077	4.443	0.379	PASS
Large SFT5 · step 1,643	48	4.22%	5.887	1.738	-0.077	4.454	0.368	PASS
Large SFT5 · step 1,833	48	1.09%	6.006	1.673	-0.091	4.434	0.470	PASS
Large SFT5 · step 2,377	48	3.54%	6.001	1.778	-0.091	4.440	0.512	PASS
Large SFT5 · step 2,444	48	3.13%	5.958	1.813	-0.073	4.420	0.481	PASS
Large SFT5 · step 3,055 — selected	48	1.23%	6.026	1.789	-0.065	4.473	0.545	PASS
Large SFT6 · step 332	48	7.90%	5.886	1.564	-0.085	4.458	0.287	PASS
Large SFT6 · step 664	48	8.99%	5.865	1.527	-0.068	4.402	0.247	PASS
Large SFT6 · step 996	48	0.95%	6.059	1.822	-0.065	4.497	0.596	PASS
Large SFT6 · step 1,328	48	4.63%	5.975	1.688	-0.079	4.482	0.442	PASS
Large SFT6 · step 1,660 — selected	48	1.36%	6.069	1.819	-0.066	4.469	0.607	PASS
Large SFT6 · step 1,661	48	4.22%	6.027	1.727	-0.080	4.483	0.518	PASS
Large Alpha RL-v1 (earlier recipe) · 2,370 updates	48	1.50%	6.357	1.873	-0.093	4.561	baseline / fixed RL	—
Large Alpha standard RL-v2 · FINAL · 2,344 updates	48	3.00%	6.282	1.814	-0.099	4.654	baseline / fixed RL	—
Small SFT6 standard RL · FINAL · 2,324 updates	48	1.23%	6.454	1.697	-0.128	4.676	baseline / fixed RL	—
Small SFT6 emotion RL · FINAL · 2,384 updates	48	2.59%	6.396	1.722	-0.092	4.449	baseline / fixed RL	—
Large SFT6 standard RL + 10% inverse-WER · FINAL · 2,360 updates	48	1.09%	6.324	1.814	-0.083	4.641	baseline / fixed RL	—
Large SFT6 emotion RL + 10% inverse-WER · FINAL · 2,378 updates	48	0.41%	6.351	1.812	-0.067	4.445	baseline / fixed RL	—

Immediate48-player saved-training-rollout preview: all six RL runs. These are intermediate-policy training generations, not the matched final-checkpoint evaluation shown below.

Listen side by side

Green: SFT6. Purple: RL. Grey: earlier SFT checkpoints. Blue: audition/reference. “No input reference” means the model did not hear the blue clip. Expand each prompt to see inline emotions, durations, pauses and bursts.

Research use and reproduction

Research and dataset augmentation are the intended uses. Obtain speaker consent, disclose synthetic audio and evaluate safety/identity on your own data. New comparison artifacts are CC BY 4.0; upstream model, software and speaker-rights terms remain applicable. See the audio repository for the frozen protocol, full score vectors and exact prompts.