8000 模型维度问题 · Issue #100 · thu-coai/CDial-GPT · GitHub
[go: up one dir, main page]
Skip to content

模型维度问题 #100

New issue
New issue
Open
Open
模型维度问题#100

Description

@terminator123
terminator123
opened on Jul 27, 2022

gpt2模型config里面n_positions=513，会报
size mismatch for transformer.h.0.attn.bias: copying a param with shape torch.Size([1, 1, 512, 512]) from checkpoint, the shape in current model is torch.Size([1, 1, 513, 513]).

改成512后，如果use_gpt2=True，会报
size mismatch for transformer.wpe.weight: copying a param with shape torch.Size([513, 768]) from checkpoint, the shape in current model is torch.Size([512, 768]).

Activity

  1. pppihf commented on Aug 9, 2022

    @pppihf
    pppihf
    on Aug 9, 2022

    我也遇到这种情况了，请问怎么解决

  2. terminator123 commented on Aug 9, 2022

    @terminator123
    terminator123
    on Aug 9, 2022
    Author

    不知道呀，你要是解决了告诉我下

  3. zhaojunGUO commented on Jan 29, 2023

    @zhaojunGUO
    zhaojunGUO
    on Jan 29, 2023

    我也是，怎么解决哇

  4. YuChuXi commented on May 13, 2023

    @YuChuXi
    YuChuXi
    on May 13, 2023
    tokenizer = tokenizer_class.from_pretrained(checkpoint, do_lower_case=False)  
model = model_class.from_pretrained(checkpoint)

    两句中加入参数 ignore_mismatched_sizes=True 然后忽略报错就好

  5. YuChuXi commented on May 13, 2023

    @YuChuXi
    YuChuXi
    on May 13, 2023

    不用动模型的配置文件

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions
      0