Repository navigation
Document and test several Buffers on one group - #31
Merged
Merged
Conversation
An integration can now import everything it needs from afdep. The environment variable that gives each Buffer its own communication stream gets a name in the library, and the benchmark uses it. Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
--lanes N builds N Buffers on the group. Lane 0 runs the existing suite beside the others, then all lanes run interleaved: every lane dispatches a layer before any lane collects it, eagerly on per-lane streams and as one captured graph. Exchange.run is split into start and finish for this. Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
State what a caller overlapping exchanges can rely on: calls enqueue and return, one Buffer runs in call order, and Buffers are independent. Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
|
Lanes builds one Buffer per lane, checks every rank asks for the same number, and gives each lane its own communication stream, so callers no longer set the stream environment variable themselves. It is set only while the Buffers are constructed and then restored. Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
--lanes N now builds its Buffers through Lanes. Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
1 of 2 tasks
Stacked on #31; only the last two commits are new. ## What and why Overlapping exchanges (DBO) needs one `Buffer` per lane, and until now the caller had to set `AFD_DEEPEP_PER_BUFFER_COMM_STREAM=1` before building them. `Lanes` does that for the caller: ```python lanes = Lanes(group=group, config=config, lanes=2) lanes[0].source_dispatch(...) # each lane is an ordinary Buffer lanes.close() ``` - Checks every rank asks for the same number of lanes. - Gives each lane its own communication stream, and raises if two lanes share one. - Closes the lanes in reverse order. - With `lanes=1` it behaves like a single `Buffer`. Which exchange runs on which lane, and the call order, stay with the caller. The env var is set only while the Buffers are constructed and then restored, because the native code reads it at construction. A constructor flag would be cleaner but is a native change; this PR has none, so the source digest is unchanged. ## Checks - [x] `python -m pytest tests/unit`, `ruff check .` and `ruff format --check .` pass - [ ] GPU suite with `--lanes 2` (BF16 and FP8, hybrid on and off). Not run yet, so this is a draft. ## Evidence Pending the GPU run: ```bash EP_DISABLE_GIN=1 torchrun --standalone --nproc-per-node=4 \ -m tests.gpu.deepep --payload fp8 --hybrid off --capacity 32 --lanes 2 ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
7355
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
A
Bufferholds one exchange at a time. To overlap several (DBO, one per micro-batch) a caller builds oneBufferper lane on the same group. That works today, but it was neither documented nor tested here.docs/api.mdgets a "Several Buffers on one group" section: theAFD_DEEPEP_PER_BUFFER_COMM_STREAM=1requirement, what a caller can rely on (calls enqueue and return, one Buffer runs in call order, Buffers are independent) and the call order every rank must follow.tests/gpu/deepep.py --lanes Nbuilds N Buffers. Lane 0 runs the existing suite beside the others, then all lanes run interleaved in DBO order, eagerly on per-lane streams and as one captured graph.Exchange.runis split intostartandfinishfor this, with the same call order as before.afdepexportsDispatchHandle, and the stream switch gets a name,afdep.deepep.PER_BUFFER_COMM_STREAM_ENV.No native changes; the source digest is unchanged.
Checks
python -m pytest tests/unit,ruff check .andruff format --check .pass--lanes 2(BF16 and FP8, hybrid on and off). Not run yet, so this is a draft.Evidence
Pending the GPU run:
EP_DISABLE_GIN=1 torchrun --standalone --nproc-per-node=4 \ -m tests.gpu.deepep --payload fp8 --hybrid off --capacity 32 --lanes 2