Repository navigation
MiniCPM5-2B sampling defaults omit repeat-penalty; without it the model runs away on a large share of generations #374
Description
Summary
The sampling table in skills/minicpm5-deploy-llama-cpp/SKILL.md (and the matching guidance in docs/deployment/llama_cpp.md) lists only --temp 1.0 and --top-p 0.95 for MiniCPM5-2B Think. With exactly those settings and nothing else, I see a very high rate of generations where the thinking channel collapses into repetition and never terminates.
Adding a single flag, --repeat-penalty, changes the outcome dramatically. Everything else was held constant across all ten runs below; only that one value changed.
HumanEval+ (164 tasks), RTX 3060 12GB
| repeat-penalty | Q8_0 score | Q8_0 runaway rate | Q4_K_M score | Q4_K_M runaway rate |
|---|---|---|---|---|
| 1.00 (as documented) | 43.3 | 55.5% | 6.1 | 92.1% |
| 1.05 | 86.0 | 7.9% | 34.1 | 61.6% |
| 1.10 | 90.9 | 4.9% | 67.7 | 25.6% |
| 1.15 | 92.1 | 1.8% | 71.3 | 11.0% |
| 1.20 | 87.1 | 5.5% | 60.1 | 7.9% |
Both quants peak at 1.15 and regress at 1.20, so this is not simply "more is better" — 1.15 looks like a real optimum rather than an artifact.
Two things stand out:
- At the documented setting, Q4_K_M is effectively unusable for coding — 92% of tasks never produce an answer at all. The quant table in the same SKILL.md describ 670D es Q4_K_M as a "small drop, ideal for laptops", which is fair at 1.15 but very misleading at 1.00.
- The lower quant is far more sensitive to this. Q8_0 recovers almost fully in a single step from 1.00 to 1.05, while Q4_K_M needs the whole sweep. So the omission hurts exactly the users the recommended quant is aimed at.
Setup
- llama.cpp served through llama-swap (
ghcr.io/mostlygeek/llama-swap:unified-cuda), RTX 3060 12GB - Official GGUFs from
openbmb/MiniCPM5-2B-GGUF, both Q8_0 and Q4_K_M -ngl 99 -c 131072 -fa on --jinja -np 4 -ctk f16 -ctv f16 -b 2048 -ub 1024 --temp 1.0 --top-p 0.95- Thinking enabled. My harness runs one repair round, so these scores are not directly comparable to standard pass@1 numbers — the relative movement is the point, not the absolute values.
- "Runaway rate" is my harness killing a generation once it detects repetition. That is my own definition, not a MiniCPM concept.
Question
Was repeat-penalty left out deliberately — for example because it costs something on a capability I did not measure? If not, would you consider adding a recommended value to the sampling table for MiniCPM5-2B? I only tested HumanEval+ on one machine, so a value validated against your own eval suite would be much better than mine.
Related: #360 covered a different gap in the same sampling guidance, so it may be worth reviewing the documented profiles as a whole rather than patching one value at a time.
Activity
Yilisameen commentedon Sep 20, 2026
Hello,
Thank you for the detailed sweep and for sharing the results here.
We have not yet run systematic experiments on repeat-penalty across the quantized models, which is why the sampling table carries no per-quant recommendation. It is something we will look into.
In the meantime, in case it is useful: on the non-quantized weights, our recommended repeat-penalty is 1.05.