Repository navigation
Tags
Tags
Support FFN ranks on every node (#27) Part of #18. Supersedes #22, whose commits are included as they were. ## What and why FFN ranks had to be ranks `[0, F)`. DeepEP numbers ranks node by node, so the FFN always landed on the first node(s). This PR lets every node hold FFN GPUs, so a deployment can scale by adding identical nodes. It also makes that layout faster. **Main changes** - **`ffn_ranks_per_domain=f`.** The first `f` GPUs of every node are FFN ranks. Callers keep real expert IDs, and the library maps them to DeepEP's numbering. - **FFN GPUs skip all cross-node work.** No tokens travel on their rails. - **Attention GPUs no longer wait at a cross-node barrier before each dispatch.** That barrier only kept a peer's next dispatch from arriving while this GPU was still combining. To make the early arrival safe: - combine signals get their own words, instead of sharing them with dispatch progress; - replies and combine staging move to the reserved area at the end of the buffer, where the next dispatch can't overwrite them. - **Hybrid off (one RDMA hop per token) also works for this layout.** It needed a fix in DeepEP's GIN barrier: a GPU could read same-node data before it had arrived. FastAFD's DeepEP M2N has the same bug. **Result** (comm-only, kermit, 4 × H200 per node, FP8): - Dropping the barrier makes per-node FFN 1.4–8.5% faster on the hybrid path. - It's 3.4–16.1% faster than FastAFD's DeepEP M2N in the same layout. - Putting the FFN on the first node(s) is still faster below 512 tokens. Per-node FFN wins at 512 tokens on 3 nodes. Details are in `docs/design.md` §20. ## Checks - [x] Unit tests and Ruff pass. - [x] GPU suite exact on 2 and 3 nodes: one to three FFN GPUs per node, BF16 and FP8, hybrid on and off. Existing layouts are unchanged, and invalid routes still assert. - [x] Fault injection with one node lagging: the change passes, and removing any one of the safety fixes makes it fail. - [x] Screens against the per-node layout before this change, FastAFD's DeepEP M2N, and today's best layout, with each arm's source digest recorded. ## Evidence `docs/validation-attention-rails.json`, plus the per-domain records from #22 (`docs/validation-per-domain*.json`, `docs/validation-direct-per-domain*.json`).