Repository navigation
Support several Buffers per process with different AFDEP_* controls - #20
Open
almogtavor wants to merge 1 commit into
Open
almogtavor wants to merge 1 commit into
almogtavor wants to merge 1 commit into
Conversation
…oexist Native code read the AFDEP_* launch controls from the process environment on every launch, while each Buffer's registered layout is sized at construction. A Buffer opened before an in-process A/B harness switched the controls then ran with the other arm's launch shapes against its own layout. Each ElasticBuffer now snapshots the AFDEP_* environment when constructed, and its constructor, dispatch, combine and config lookup read that snapshot. Adds tests/gpu/two_buffers.py covering same and different configs, eager and in graphs, and documents multiple Buffers per process in docs/api.md.
This was referenced Oct 26, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #5.
Problem
Native code read the
AFDEP_*launch controls from the process environment on every launch (deep_ep::get_env), while each Buffer's registered layout is sized at construction. An in-process A/B harness that opens Buffer A, switches the controls and opens Buffer B leaves A running with B's launch shapes against A's layout. WithAFDEP_UNION_BUFFERS=1on B this fails every rank at thebuffer.hpp:1059buffer-size assertion; with other controls it can silently mismatch or fault.Fix
csrc/utils/system.hpp:get_envreadsAFDEP_*names from a thread-local snapshot when one is active (LatchedEnvScope), otherwise from the environment as before.csrc/elastic/buffer.hpp: eachElasticBuffersnapshots theAFDEP_*environment at construction and activates it in the constructor,dispatch,combineandget_afd_sparse_hybrid_config.LOCAL_CHANGES.jsonupdated for both vendored files.docs/api.md: several Buffers per process are supported; every rank enqueues their exchanges in one order; each Buffer keeps the controls it was opened with.docs/design.md(R10): explains the old in-process A/B failure.tests/gpu/two_buffers.py: opens A under the process env and B under--env-b KEY=VALUE, then checks eager alternation, one graph per Buffer replayed alternately, one graph interleaving both, and two graphs on one Buffer.Validation (CoreWeave, 4x H200,
EP_DISABLE_GIN=1, 2 attention + 2 FFN ranks)--env-b AFDEP_UNION_BUFFERS=1buffer.hpp:1059assertion on all ranks--env-b AFDEP_KEEP_ENTRY_BARRIERS=1--env-b AFDEP_SHARED_FFN_RECEIVE=1--env-b AFDEP_SHARED_ROLE_KERNELS=1--env-b AFDEP_UNION_NOTIFY=1Checked exchanges per rank: eager 24, graph_alternating 96, graph_interleaved 96, two_graphs_one_buffer 96. Native source digest
4ceca478....Also on this PR:
tests.gpu.deepepbf16 and fp8 pass, Ruff clean, unit tests (48) pass.