8000 [Feature Request]: how to use megatron of ms-swift to finetune minicpm5-2b · Issue #380 · OpenBMB/MiniCPM · GitHub
[go: up one dir, main page]

Skip to content

[Feature Request]: how to use megatron of ms-swift to finetune minicpm5-2b #380

Open

Description

@haodongze

Feature request / 功能建议

how to use megatron of ms-swift to finetune minicpm5-2b

Activity

  1. hansjohn commented on Sep 21, 2026

    @hansjohn
    Collaborator

    Hi, thanks for your interest in MiniCPM5-2B!

    MiniCPM5-2B can be fine-tuned with Megatron-SWIFT. For better alignment with the MiniCPM5-2B training format, we recommend using the minicpm5_2b template available in ms-swift>=4.6.0.dev0.

    The following installation example uses ms-swift==4.6.0.dev0:

    git clone https://github.com/modelscope/ms-swift.git
    cd ms-swift
    git checkout 654e24f17b5d9f40ed4b9ee4c56a723320244db5
    pip install -e .
    pip install "transformers>=5.6,<5.17"
    pip install mcore-bridge==1.6.4

    For MiniCPM5-2B, use:

    megatron sft \
        --model /path/to/MiniCPM5-2B \
        --model_type llama \
        --template minicpm5_2b \
        --dataset /path/to/train.jsonl \
        --finetune true \
        --output_dir /path/to/output \
        ...

    The important arguments are:

    --model_type llama
    --template minicpm5_2b
    

    We have also updated the documentation with the ms-swift and Megatron-SWIFT configuration:

    https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/ms_swift.md

    For the complete training options, please refer to the official ms-swift documentation:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0