Repository navigation
MiniCPM5-2B 在规则密集任务上产生循环思考的现象 (21/81例) #387
Description
描述
我们在一个多轮对话任务集上批量测试了 MiniCPM5-2B (bf16,开思考), 每个任务要求模型输出一个工具调用 JSON. 然而有 21/81 用例卡在思考循环直至达到 --max_tokens=8192, 下面简要描述失败案例, 并想请教这是否有推荐缓解措施.
测试环境
- 引擎:昇腾 910B4, vLLM-Ascend
- 采样:
--temp=1.0, --top_p=0.95(文档推荐), 思考模式 --max_tokens=8192, 每个任务跑一次, 共 81 个任务
解释
卡住用例模式大致分两种:
- 模式 A. 一句话原样复读(极端情况), 即模型重复引用系统提示里的某条规则直到被截断:
- 等等,规则7说:"缺失回复按参数数组顺序覆盖全部客户缺失 key,并带上
description 明示的必要格式、单位或可选值;不得询问已锁定项和系统待处理项。"- 等等,规则7说:"……"
- 模式 B. 逻辑循环 (更常见), 即模型生成答案 → 怀疑 → 重新推导 → 生成答案. 在约 5 到 49 句之间循环,重复 4 到 77 次:
所以输出应该是:
{"name":"set_config","arguments":{"额定功率":58.504,"额定电量":38.267}}
等等,规则里说"value 是唯一非空标量时表示该 key 已锁定"。所以这些 key 还没锁定,
需要输出 set_config。等等,规则又说"必须从参数数组第一项扫描到最后一项……"
(如此循环往复)
常见的触发条件是规则多且互相牵扯的提示词 (我们的系统提示规定了冲突参数、缺失参数分别怎么处理,哪条规则优先等), 多轮历史时更容易出现. 卡住用例每次耗时 300~333 秒, 正常用例远低于这个时间.
请问
- 是否你们内部有在 Agent 类或规则密集任务上, 评估过模型出现"思考不收敛"现象的占比, 这将有助于我们评估模型部署相关风险
- 除了对参数
--repeat-penalty的设置 (我们有看到 MiniCPM5-2B sampling defaults omit repeat-penalty; without it the model runs away on a large share of generations #374 里你们对非量化权重的推荐值 1.05), 是否有其他验证过有效的缓解方法? - 我们还准备在同一套服务栈上部署 MiniCPM5-2B 的 INT8 (W8A8) 模型, 不知道你们对于量化权重有没有推荐的配置策略?
Activity
Lmeng486 commentedon Oct 20, 2026
Related data point that partly bounds the "raise the budget" mitigation discussed in #388 — posting it here because it cuts against my own conclusion, which seems relevant to your question 2.
On my side (MiniCPM5-2B Q4_K_M GGUF, phone,
max_tokens=384): the loop failures were pure budget exhaustion — the whole 384-token budget was consumed insidereasoning_content,contentcame back empty (8/27 positive cases). Raisingmax_tokens384 → 512 took that from 8/27 to 1/9. So for that failure mode, more budget was the fix.But your cases loop at
--max_tokens=8192, which means "just give it more budget" is not a general answer. It only works when the loop is bounded by the budget rather than genuinely non-convergent. Your Pattern B (generate → doubt → re-derive → generate) reads like the latter, so I'd expect sampler-side knobs and prompt structure to matter more thanmax_tokensthere. Treat my result as scoped to "short-budget truncation", not as a remedy for non-convergent thinking.One config detail that did matter on my side: llama.cpp's default
min_p=0.05filters out the low-probability tokens needed to escape a loop; I had to setmin_p=0.0explicitly for runs to converge. I don't know whether vLLM-Ascend exposes an equivalent sampling floor, but if it does, it may be worth checking before tuningrepeat_penaltyalone.Data if useful: the measurement is #388, raw TSVs available on request. Not claiming it solves your case — the two setups differ in both budget scale and engine.