You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched acco
8000
unts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Feature Request]: how to use megatron of ms-swift to finetune minicpm5-2b #380
MiniCPM5-2B can be fine-tuned with Megatron-SWIFT. For better alignment with the MiniCPM5-2B training format, we recommend using the minicpm5_2b template available in ms-swift>=4.6.0.dev0.
The following installation example uses ms-swift==4.6.0.dev0:
Activity
hansjohn commentedon Sep 21, 2026
Hi, thanks for your interest in MiniCPM5-2B!
MiniCPM5-2B can be fine-tuned with Megatron-SWIFT. For better alignment with the MiniCPM5-2B training format, we recommend using the
minicpm5_2btemplate available inms-swift>=4.6.0.dev0.The following installation example uses
ms-swift==4.6.0.dev0:For MiniCPM5-2B, use:
megatron sft \ --model /path/to/MiniCPM5-2B \ --model_type llama \ --template minicpm5_2b \ --dataset /path/to/train.jsonl \ --finetune true \ --output_dir /path/to/output \ ...The important arguments are:
We have also updated the documentation with the ms-swift and Megatron-SWIFT configuration:
https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/ms_swift.md
For the complete training options, please refer to the official ms-swift documentation: