8000 Released LoRA checkpoints lack the per-dataset displacement buffers needed by decode_positions · Issue #1 · pairlab/flomo · GitHub
[go: up one dir, main page]

Skip to content

Released LoRA checkpoints lack the per-dataset displacement buffers needed by decode_positions #1

Open

Description

@rakhimovv

Hi, thanks for releasing FloMo and the checkpoints.

I tried to decode motion-token predictions to 3D positions with the released LoRA checkpoints from shitijgovil/FloMo and could not, because the per-dataset displacement scale buffers are not in the files.

What I see

  • ma/flomo_ma.pt holds 614 tensors under model and vma/flomo_vma.pt holds 616; neither contains per_ds_cumulative_disp_pct_low / per_ds_cumulative_disp_pct_high (nor the other per_ds_* buffers registered in flomo/models/modalities.py:449-456).
  • So after loading, those buffers keep their empty initial shape (0, 3, 1, 1, 1), and MotionTokensWanPretrainedModality.decode_positions (modalities.py:691) fails at the index_select on line 710.
  • The top-level normalizer_stats entry holds the action normalizer (molmoact2, realyam: p02/p98 of eepose_6drot), not these track displacement percentiles.

Likely cause

save_checkpoint in flomo/utils/train_utils.py:298-300 keeps, when lora_enabled, only state-dict keys that start with a requires_grad parameter name. Buffers are never parameters, so they are dropped from every LoRA checkpoint.

Ask

Could you publish the track_stats used for training (cumulative displacement low/high percentiles per dataset: egoverse, molmoact2, realyam, realyam_human), or checkpoints saved with these buffers? For future checkpoints, adding model.named_buffers() to the LoRA branch of save_checkpoint would keep them.

Checked against main at 5938922.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0