8000 k must have shape (batch_size, seqlen_k, num_heads_k, head_size_og) · Issue #5 · zai-org/Inf-DiT · GitHub
[go: up one dir, main page]

Skip to content

k must have shape (batch_size, seqlen_k, num_heads_k, head_size_og) #5

Open

Description

@rainsoulsrx

When I set --infer_sr_scale 1, there is an error, but when set infer_sr_scale=2, run correct, why?

Activity

  1. rainsoulsrx commented on May 21, 2024

    @rainsoulsrx
    Author

    and in the code, collect_attention is set False, the following set is not taking effect."You can change the "inference_type"(line 27 in generate_sr_big_cli.sh) to "ar"(parallel size=1), "ar2"(parallel size = block_batch(line 28)) or "full"(generate the entire image in one forward)."

  • yzy-thu commented on May 22, 2024

    @yzy-thu
    Contributor

    collect_attention is a debug parameter, irrelevant to inference. Why you need infer_sr_scale==1? I haven't tested this.

  • Dorniwang commented on May 28, 2024

    @Dorniwang

    yeah, it doesn't work, when i specify the inference_type=ar2 manually, it will give this error

  • Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      No labels
      No labels

      Type

      No type

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions

        0