8000 Document and test several Buffers on one group by Etelis · Pull Request #31 · afdEP/afdEP-comm · GitHub
[go: up one dir, main page]

Skip to content

Document and test several Buffers on one group - #31

Merged
Etelis merged 6 commits into
mainfrom
buffer-integration-api
Nov 15, 2026
Merged

Etelis merged 6 commits into
mainfrom
buffer-integration-api

Conversation

@Etelis
@Etelis Etelis commented Nov 15, 2026
Copy link
Copy Markdown
Member

What and why

A Buffer holds one exchange at a time. To overlap several (DBO, one per micro-batch) a caller builds one Buffer per lane on the same group. That works today, but it was neither documented nor tested here.

  • Docs. docs/api.md gets a "Several Buffers on one group" section: the AFD_DEEPEP_PER_BUFFER_COMM_STREAM=1 requirement, what a caller can rely on (calls enqueue and return, one Buffer runs in call order, Buffers are independent) and the call order every rank must follow.
  • Test. tests/gpu/deepep.py --lanes N builds N Buffers. Lane 0 runs the existing suite beside the others, then all lanes run interleaved in DBO order, eagerly on per-lane streams and as one captured graph. Exchange.run is split into start and finish for this, with the same call order as before.
  • Exports. afdep exports DispatchHandle, and the stream switch gets a name, afdep.deepep.PER_BUFFER_COMM_STREAM_ENV.

No native changes; the source digest is unchanged.

Checks

  • python -m pytest tests/unit, ruff check . and ruff format --check . pass
  • GPU suite with --lanes 2 (BF16 and FP8, hybrid on and off). Not run yet, so this is a draft.

Evidence

Pending the GPU run:

EP_DISABLE_GIN=1 torchrun --standalone --nproc-per-node=4 \
    -m tests.gpu.deepep --payload fp8 --hybrid off --capacity 32 --lanes 2

An integration can now import everything it needs from afdep. The
environment variable that gives each Buffer its own communication stream
gets a name in the library, and the benchmark uses it.

Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
--lanes N builds N Buffers on the group. Lane 0 runs the existing suite
beside the others, then all lanes run interleaved: every lane dispatches a
layer before any lane collects it, eagerly on per-lane streams and as one
captured graph. Exchange.run is split into start and finish for this.

Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
State what a caller overlapping exchanges can rely on: calls enqueue and
return, one Buffer runs in call order, and Buffers are independent.

Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
@github-actions
github-actions Bot commented Nov 15, 2026 •
Copy link
Copy Markdown
PR Preview Action v1.8.2
Preview removed because the pull request was closed.
2026-11-15 14:51 UTC

Lanes builds one Buffer per lane, checks every rank asks for the same
number, and gives each lane its own communication stream, so callers no
longer set the stream environment variable themselves. It is set only
while the Buffers are constructed and then restored.

Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
--lanes N now builds its Buffers through Lanes.

Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
@Etelis Etelis mentioned this pull request Nov 15, 2026
1 of 2 tasks
Stacked on #31; only the last two commits are new.

## What and why

Overlapping exchanges (DBO) needs one `Buffer` per lane, and until now
the caller had to set `AFD_DEEPEP_PER_BUFFER_COMM_STREAM=1` before
building them. `Lanes` does that for the caller:

```python
lanes = Lanes(group=group, config=config, lanes=2)
lanes[0].source_dispatch(...)  # each lane is an ordinary Buffer
lanes.close()
```

- Checks every rank asks for the same number of lanes.
- Gives each lane its own communication stream, and raises if two lanes
share one.
- Closes the lanes in reverse order.
- With `lanes=1` it behaves like a single `Buffer`.

Which exchange runs on which lane, and the call order, stay with the
caller.

The env var is set only while the Buffers are constructed and then
restored, because the native code reads it at construction. A
constructor flag would be cleaner but is a native change; this PR has
none, so the source digest is unchanged.

## Checks

- [x] `python -m pytest tests/unit`, `ruff check .` and `ruff format
--check .` pass
- [ ] GPU suite with `--lanes 2` (BF16 and FP8, hybrid on and off). Not
run yet, so this is a draft.

## Evidence

Pending the GPU run:

```bash
EP_DISABLE_GIN=1 torchrun --standalone --nproc-per-node=4 \
    -m tests.gpu.deepep --payload fp8 --hybrid off --capacity 32 --lanes 2
```
@Etelis
Etelis marked this pull request as ready for review November 15, 2026 14:49
@Etelis
Etelis requested a review from almogtavor as a code owner November 15, 2026 14:49
@Etelis
Etelis merged commit 510172c into main Nov 15, 2026
5 checks passed
@Etelis
Etelis deleted the buffer-integration-api branch November 15, 2026 14:49
@Etelis Etelis mentioned this pull request Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0