Repository navigation
feat(switch): deliver each accepted box connection to the proxy with the box's identity - #1810
Conversation
… ran out the coder's uncommitted work as it stood when its 7200s ran out The coder was stopped at the end of its time with these changes not committed. The runtime committed its working tree as it stood so the work is kept: this commit is a snapshot, not a finished step, and may not build or pass its tests.
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
The fixtures built TCP and UDP segments with ChecksumCapabilities::ignored(), leaving the TCP checksum zero. The peer's stack now verifies inbound TCP checksums at parse (BepDevice::capabilities advertises Both), so those segments were dropped before the stack could answer them: three T43-era tests expected a reset or a port-unreachable that never came. A real box kernel never sends a zero TCP checksum, so the fixtures now emit with the default capabilities, which fill every checksum. peer_binds_no_socket also counted the ARP reply among the answers its poll produces, but arp_exchange — which the pool rewrite added to the test — consumes that reply while teaching the stack its neighbour. The count is back to the two answers the poll actually produces: the reset and the port-unreachable.
The pool in switch::bep_host delivers a box's connection by dialing the proxy's unix socket, token first, then the fixed header naming the box's own switch address. The supervisor now carries the other end of that wiring: it mints the per-boot token beside the marker nonce, derives the proxy socket path beside the switch and gate sockets, names the recorded per-source cap, and hands the peer its wire in-process. Until the proxy's own acceptor lands, a stand-in accepts in its place: crates/minvmd/src/net/bep_stub.rs, a test/e2e surface the supervisor binds only when MINVMD_BEP_STUB is set — the e2e lane's flag, nothing else's. It takes the same token over its start-up channel, checks every connection in order — the peer's uid against the daemon's own, then on Linux the peer's pid against the host daemon's (SO_PEERCRED, read before any presentation byte), then the token, then the header version — and refuses and audits on any mismatch, closing without an answer so nothing refused is ever presented as a box. A connection that passes is answered with one line naming the source it was presented from. The registry's rows become the pool's partition through RegisteredBoxes over the table's live view, so a row that lands grows its box's share within a stack turn. HostGvproxy::spawn threads the wire and the box source to the peer beside the switch runtime; its tests pass a socketless source and an acceptor path nothing listens at, which is the acceptor-down answer the pool gives on a production boot.
Annotate the requirement with the working values the switch and the host daemon ship: the proxy's listener on port 8118, a per-source cap of 16 concurrent connections from one box, and the delivery's listener pool partitioned per registered box so each box's share is that cap, added at its registration and withdrawn with its row. All proposed working values, to be tuned by the integration run.
Add the NET-132 case proxy_sees_each_vm_box_by_its_switch_address: activate two boxes, connect each to the proxy's address through the switch, and read the delivery's source at the stand-in acceptor the host daemon mints a per-boot token for when MINVMD_BEP_STUB is set. Each box's answer must name its own switch address and the proxy's endpoint, and neither box's answer may name the other's address. The case is gated on the switch (MINVMD_GVPROXY_BIN) like its T43 neighbours, so the no-switch native lane skips it, and it runs last in the whole-lane order because it restarts the daemon to hand the stub its token.
Straighten the import order in bep_stub.rs and wrap the RegisteredBoxes projection so both files pass the fmt-check gate the linux native lane runs.
Spec lint
Advisory. None of this blocks the merge; the review decides. Advisory, and never a merge gate. Linter: |
|
Round 1 of 10 on a failed check suite, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. A box that connects to the proxy's address is delivered to the proxy's unix socket over one connection per flow, with the delivery header naming the box's own switch address as the source and the acceptor's peer carrying the daemon's uid and pid and the per-boot token; at the stand-in acceptor two boxes' connections are read back each naming its own switch address and the proxy's endpoint, a same-uid host process presenting the wrong token or header version is refused and audited, never presented as a box, a box past its per-source cap of 16 is reset while its sibling keeps its share, and a stalled acceptor holds one box's flow while the others keep moving. Notes from the agent: All three failed checks were the uncommitted snapshot at the old head: the switch pool tests' fixtures advertised ignored checksums, which smoltcp rejects at inbound TCP parse (fixed in aca06cb, including peer_binds_no_socket counting the two frames the turn really emits — the ARP reply is consumed inside arp_exchange), and minvmd did not compile against the new BepPeer::spawn arity (fixed in 6df2447). Out of the task's layers and why: crates/minvmd/src/net.rs — HostGvproxy::spawn forwards the wire and the box source to BepPeer::spawn, so its signature grew those arguments and its five in-file test call sites gained an idle wire and an empty box source; it also declares pub mod bep_stub and re-exports BepBoxSource/BepWire, because the non-libkrun build the |
The macOS unit lane runs `cargo clippy -p minvmd --all-targets -- -D warnings` and then the minvmd suite, and both failed there, for two reasons one cfg short of each other: - `AsRawFd` was imported unconditionally but used only inside the `#[cfg(target_os = "linux")]` peer-credentials read, so off Linux the import read as unused and `-D warnings` turned it into an error. - `host_process_never_arrives_from_a_box_address` asserted the pid refusal end to end, and only `SO_PEERCRED` can prove that: off Linux `peer_credentials` names no pid, so the foreign-process presentation was answered rather than refused, and the answer-empty assertion fell. Gate both to Linux — the layer binds the pid check to Linux in the same breath — and reword the test's doc comment so the pid leg is named as Linux's. The uid check, the per-boot token and the header version are portable refusals and stay unconditional. Verified by inverting the file's cfgs and re-running clippy `-D warnings` and both delivery tests under the non-Linux semantics, then restoring them.
|
Round 2 of 10 on a failed check suite, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. Two VM boxes' connections to the proxy's address arrive at the proxy's unix socket from their own switch addresses in the delivery header, a same-uid host process without the per-boot token is refused and never presented as a box, a box past its per-source cap is reset while its siblings keep their shares, and a stalled acceptor stalls one box's flow while the others keep moving — and since this round the stand-in's Linux-only SO_PEERCRED pid check no longer breaks the off-Linux build, so the macOS lane's clippy and minvmd suite see the same proof rather than a red. Notes from the agent: Round 2: the two failed checks are both the ci-macos.yml No darwin target on this box, so the off-Linux path was verified by inverting the file's cfg attributes, running clippy -D warnings and both minvmd delivery tests under the non-Linux semantics (all clean), then restoring. The e2e line above exited 0 with the case SKIPPED: this box has no gvproxy and no /dev/kvm, so there was no switch to deliver over. The lanes that run it for real are ci-linux-kvm and the ci-macos e2e job; ci-linux-kvm-success was green in this round's failed-check sui…[truncated] |
The NET-132 case handed socat /dev/null as its left address — the shape the MAC and ARP cases use, which read only socat's stderr and the box's ARP table — so the stand-in's answer line went to /dev/null and the case's greps always found nothing. Give the probe an empty pipe as socat's left address with -t 10, the shape the hostname-proxy case's bogus-head probe uses, so the delivered line lands on the box's stdout. Also take the case's two seed dirs into the EXIT trap, since every failure path in it goes straight to fail.
The teardown's new rm -rf lines named two variables nothing assigns until the case runs, and the script runs under set -u: every run aborted the EXIT trap with 'unbound variable' and skipped the workdir cleanup, even on paths the case never touches. Initialise both to empty beside the lane's other seed dirs, so the trap sees them on every path and the failure paths' dirs are the only ones it removes.
smoltcp folds a FIN that arrives with the handshake-completing ACK straight from SynReceived into CloseWait, and the daemon's turns are a hundred milliseconds apart, so a box that finished sending as it connected - the e2e probe's shape - always reached the service pass as CloseWait. The pool only dialled Established sockets, so that connection hung: never delivered, never reset, its slot lost to the share. Deliver CloseWait accepted connections too. The FIN only closed the box's write side: smoltcp still lets the socket transmit, so the acceptor's answer still reaches a box that can only listen. Pinned by a_half_closed_connection_is_still_delivered, which queues the box's FIN behind its handshake ACK between turns; without the fix the acceptor takes nothing.
…-accepted-box-connection-to-the-proxy # Conflicts: # crates/minvmd/src/cmd/run.rs
The merge from main brought T63 (#1788), whose register_node_namespace now takes the node's proxy and answerer ports as its admitted pair. bep_stub.rs still called the zero-argument form, so the minvmd lib test target — both of this task's test lines and the agent-verify gate — failed to compile at all. The test now registers the node row the way cmd/run.rs does at VM boot, with the default pair (7654, 7656) the assignment resolves when no operator override pins it — the same literals every sibling test in the crate uses.
|
Round 3 of 10 on a failed check suite, pushed as Resolved conflicts in:
Verify Outside the task's
Converted to a draft: something above needs a person before review. On this branch a box's proxy-bound connection is delivered even when the box finished sending as it connected: the pool hands a CloseWait accepted socket to the dialer with the same delivery header, so two VM boxes still arrive at the proxy's unix socket each from its own switch address and the e2e case now reads the answer line its greps assert on instead of discarding it with /dev/null. A same-uid host process without the per-boot token is refused by the stand-in and never presented as a box, a box past its cap is reset while siblings keep their shares, and a stalled acceptor holds one box's flow while others keep moving - the four pool tests and both stand-in tests all green on this tree. Notes from the agent: Both red checks (ci-macos-success, e2e) come from the macOS lane's e2e job, whose session-e2e step is the only one that runs the new case with MINVMD_GVPROXY_BIN set. Three defects fixed this round, commits 5f03111, c57b849, 9e0e8a7. (1) The case's probe read as |
|
/revise
|
The pool's slots are anonymous, so smoltcp assigned inbound SYNs to any listening socket in arrival order: a box that opened more connections than its share in one turn took a sibling's listener, and the sibling's same-turn SYN met no socket at all and was reset. BepStack::poll now pre-screens the inbound frames before the interface sees them and refuses a connection-opening SYN to the proxy port whose source has no row or already holds its per-source cap, answering each refusal with a reset the box's own stack accepts and one rate-limited warn naming the source and the reason. A retransmitted SYN of a connection the pool already carries passes through, so the cap never counts one connection twice. one_boxs_exhaustion_never_starves_a_sibling now drives the burst it always meant to: box A opens cap + 2 connections and box B opens one before a single turn, and B is still delivered. The peer also never emits an Ethernet broadcast again: an outbound ARP request's destination is rewritten to the switch-derived MAC of the address it asks about, so a 61-second neighbour expiry re-resolves over the point-to-point lane instead of reaching every box on the plan, and a_re_resolved_neighbour_is_aimed_not_broadcast pins it. pump_to_acceptor now notifies the stack after each successful write, so a drained channel reopens the box's window without waiting for the 100 ms tick. The module docs carry the gate sentence again — T69 is the only task that may bind a socket on the peer, and until the credentialed-only gate rule (#1665) lands, the supervisor hands the peer the new empty source, NoBoxes.
The supervisor hands HostGvproxy::spawn the new empty source unless the stand-in is enabled: a socket in the pool is a lane onto the host for the box whose share holds it, and the credentialed-only gate rule (#1665) is still ahead, so rows registered or not — the guest node namespace's row included — a production boot partitions nothing and the pool binds nothing, leaving a box's connection to the proxy's address to the stack's own reset. Only MINVMD_BEP_STUB, the e2e lane's flag, wires the table's box rows. production_wiring_leaves_pool_len_zero_ with_rows_registered pins both wirings: zero with rows registered, the per-source cap per box row when the stand-in asked for them. The node namespace's row is the guest's own root netns, the daemon's tap — never a box — so RegisteredBoxes excludes the subnet's daemon address from the rows it hands the pool: it sits outside the hand-out run every client box is allocated from, so excluding that one address names exactly the node row. delivered_connection_arrives_from_the_boxs_ switch_address now asserts the node row buys no share and that a connection from the daemon address is reset and presented as nothing. Off Linux the stand-in reads the peer's uid with getpeereid instead of handing back its own, so the uid check is real on macOS and host_process_never_arrives_from_a_box_address covers it; no pid comes with it, so the pid check stays the Linux extra. The per-boot token is compared in constant time: a byte-at-a-time compare would leak how many leading bytes a guess got right through when the refusal closes the connection. test_util re-exports smoltcp's tcp State so a test in another crate can name it.
The proxy-source case stops only the session daemon before exporting MINVMD_BEP_STUB, so on a VM lane the MAC and ARP cases above leave a minvmd running with no stub flag, the activation finds it, and the whole case runs against a daemon that bound no stand-in and no pool: the boxes' connections to the proxy's address were reset and the case died in a blank answer that named no cause. The case now stops both, the pair teardown stops — min stop, and minvmd stop on a VM lane — before the flag goes out, so the activation autospawns the one daemon that carries it. The case also asserts the stand-in's socket exists before any box probes: the daemon binds it at boot only under the flag, at the path beside the switch socket, so a daemon that came up without it fails there, with the cause named. Every failure path now prints the host daemon's own log tail first — the autospawned minvmd writes a daily `minvmd.log.<date>` whose dated name the global fail()'s `*.log` pattern never reaches — through bep_fail, which drops the flag, names that log, then the global diagnostics.
|
Round 4 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. Two VM boxes connecting to the proxy's address are each presented at the proxy's unix socket from their own switch address in the delivery header, with a sibling's same-turn SYN still delivered and answered beside a box that has exhausted its share (the pool now refuses a connection-opening SYN before smoltcp assigns a listener), a stalled acceptor holding only its own flow while the others keep moving, a same-uid host process refused and never presented as a box (the token compared in constant time, the peer uid read with getpeereid off Linux), the guest node namespace's row buying no share and a connection from the daemon address reset — and a production boot binds no pool socket at all until the credentialed-only gate rule (#1665) lands, while the e2e case that proves the delivery end to end now stops the host daemon before its stand-in flag goes out, asserts the stand-in's socket exists before probing, and names the minvmd log when it fails. Notes from the agent: All ten directive items addressed; this round's commits are 9d1ee35 (switch layer), 9b7ce8f (minvmd layer) and b3aa861 (e2e script), touching only layer files. (1) The e2e evidence the directive demands is the macOS lane's e2e job at the pushed head — this box has no KVM/gvproxy, so the local run of the line exits 0 with the case SKIPPED; the case the lane runs now also carries the three diagnostics the directive asked for (minvmd stop before the flag, test -S on the stand-in socket, minvmd.log tail on failure), which addresses the inherited-daemon cause a lane failure would most plausibly trace to. (2) The libkrun-gated supervisor wiring was verified with a fake LIBKRUN_PREFIX pointing at a touched libkrun.so: the build script emits cargo:rustc-cfg=minvmd_libkrun (confirmed in the build output) and cargo clippy -p minvmd --all-targets is clean under it; the normal (gate-off) config is clippy-clean too and is what the passing test lines compiled. (3) Out-of-layer files on this branch fro…[truncated] |
A daemon booted under MINVMD_BEP_STUB exists to be probed through the box egress proxy's switch, and the stand-in's socket is bound before that switch is spawned — so a degraded boot looks healthy to anything that waits on the socket, and every probe below it fails in terms of a peer that was never there. The macOS lane's e2e run showed exactly that shape: activations healthy, socket present, the box's connection to the proxy's address refusing, with nothing in the annotation stream naming the leg that failed. Fail the boot when the switch will not come up while the flag is set (the e2e lane's and nothing else's — production boots keep the warn-and-degrade behaviour), so the cause surfaces at the activation that triggered it instead of three steps later.
…mplete The macOS lane failed this case twice on a healthy-looking boot: the activations succeeded, the stand-in's socket existed, and the box's connection to the proxy's address still did not complete — with socat's stderr and the daemon's log tail printed as plain text, none of which reaches the run's annotation stream, so the only thing the lane said from outside was the probe's own failure. Three moves, all inside the case: - gate the probes on the daemon's own log line for the peer, scoped to the lines this case's daemon wrote (the host log is shared by every daemon of the run; an unscoped grep reads the earlier cases' daemons' identical switch-up lines), so a boot whose switch never came up fails here naming the spawn, not at a box's blank answer; - when a probe fails, emit the deciding facts as annotations: the probe's stderr, the box's ARP entry for the proxy address, a same-box cross-probe to an unlistened port (its RST proves the box-to-peer path and isolates the failure to the pool's listener or the delivery's dial), and the daemon's own box-egress-proxy lines; - the answer-grep failures carry the same daemon-log annotation. Pair commit: the minvmd change makes the switchless boot these guard against fail loudly at boot.
|
Round 5 of 10 on a failed check suite, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. The pool, cap, refusal and stall properties NET-132 names are pinned by the six switch and minvmd tests above, and the end-to-end case (two VM boxes arriving at the proxy's unix socket from their own switch addresses, cross-box absence asserted) is green on the Linux KVM lane at both prior heads — while this round makes the macOS lane, where that case failed twice with only a probe annotation to read, unable to fail silently any more: a daemon booted under MINVMD_BEP_STUB now refuses to come up switchless instead of binding the stand-in and looking healthy, and any probe failure arrives as annotations naming the failed leg (socat's stderr, the box's ARP entry for the proxy address, a same-box cross-probe whose RST isolates the box-to-peer path from the pool's listener, and the daemon's own box-egress-proxy log lines, scoped to this case's daemon). The macOS re-run of the case is what a person will see next on the lane; it has not run at this head yet. Notes from the agent: The lane failure (run 37027254317: 'box A's connection to the proxy's address did not complete', macOS-only; KVM green at both heads, and the macOS-hosted unit job green on the same commits) could not be reproduced on this box — no macOS, no gvproxy binary, no KVM — so this round made the case name its own cause rather than guessing a fix. Two things it lands: (1) crates/minvmd/src/cmd/run.rs: under MINVMD_BEP_STUB (the e2e's flag, nothing else's) a HostGvproxy::spawn failure is now fatal instead of warn-and-degrade. This closes the one silent shape the analysis proved: the stand-in socket is bound before the switch is spawned, the case's readiness assert waited on that socket, MINVMD_VM_OWN_IP (the only thing that made a spawn failure fatal) is set nowhere in this repo, and the seed project declares no session packages, so a switchless daemon would still activate boxes and read fib_trie fine — every observation the failed run left is consistent with that shape, and it cannot be disti…[truncated] |
…-accepted-box-connection-to-the-proxy Conflicts: docs/specs/18-spec-box-networking/18-spec-box-networking.md
The lane exports RUST_LOG=warn,minimald::exec=info for the whole script, and the minvmd this case autospawns builds its EnvFilter from that value, which drops every INFO record from minvmd's own modules — so the gate's 'host gvproxy switch up; box egress proxy peer started' line could never appear and the case timed out and failed on every lane that runs it. Widen the filter for this case's daemon only, restoring it on every exit, and hold the gate on both records: the stand-in's flag-gated line proves MINVMD_BEP_STUB reached this daemon, not just that the socket file exists, and the peer's line proves the switch came up behind it. The widened value also lets the pool's per-delivered-connection DEBUG line through, so the log tail a failure prints names the surface the case observes.
The macOS lane runs the whole session e2e without E2E_MINIMAL_ARGS, so every mnl call reaches min without --provider local-minvmd. On macOS the CLI resolves the minvmd backend either way (client_provider_kind gates on the OS), so all earlier cases pass — but GlobalArgs::use_minvmd() is a strict flag check, and it gates the box registration and its withdrawal against the VM host daemon (T66). Without the flag the proxy-source case's own_ip boxes activate unregistered: the guest self-allocates its switch address from the reserve (100.64.0.2 in the failed run), the box-egress pool has no row to partition a share from, and the pool's prescreen resets the box's connection to the proxy's address — 'reset a connection from a source with no registered row' — with nothing on the CLI's stderr naming the cause. The KVM lane passes the flag and is green through the whole suite, so the script now supplies the same args for its one VM lane that cannot: an explicit E2E_MINIMAL_ARGS is kept verbatim and Linux is untouched, so the native lane keeps its backend.
…-accepted-box-connection-to-the-proxy
|
Round 7 of 10 on a failed check suite, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. Two VM boxes connecting to the proxy's address arrive at its unix socket from their own switch addresses in the delivery header, a same-uid host process without the per-boot token is refused and never presented as a box, a box past its cap is reset while its siblings keep their shares, and a stalled acceptor holds one box's flow while the others keep moving — and the proxy-source case that proves it now runs on the macOS VM lane too, with its boxes registered with the host daemon, because the script drives that lane's CLI with the same --provider local-minvmd the green KVM lane has always passed. Notes from the agent: Round 7's two failed checks (the macOS e2e job and its ci-macos-success aggregator) were one defect, now fixed in the e2e layer. The macOS lane invokes the script with no E2E_MINIMAL_ARGS, so every min call there ran without --provider local-minvmd; the macOS CI workflow is frozen and out of scope, so the default lives in the script. On macOS the CLI resolves the minvmd backend either way (client_provider_kind gates on the OS), which is why every earlier case passed — but GlobalArgs::use_minvmd() is a strict flag check, and it gates the T66 box registration and withdrawal, so the proxy-source case's own_ip boxes activated UNREGISTERED: the guest self-allocated 100.64.0.2 from the reserve, the box-egress pool had no row to partition a share from, and its prescreen reset the box's connection to the proxy's address ('box egress proxy: reset a connection from a source with no registered row', source 100.64.0.2 — the failed run's check-run annotations) with nothing on the CLI's stderr naming the cause. The KVM lane passes the flag and was green through the whole suite at the same head, so the script now defaults E2E_MINIMAL_ARGS to --provider local-minvmd on Darwin when the caller passed none: an explicit value is kept verbatim and Linux is untouched (verified locally: the native run built minimald, injected nothing, and the case self-skipped).…[truncated] |
|
/revise Round 8. The macOS e2e now passes at 3bf6513 and the 13:40Z asks 1-9 are met; these are the architecture ruling on the neighbour and three pool holes, never sent before.
The PR body is edited by hand; do not rewrite it. |
The neighbour architecture ruling (review): the peer answers its own stack's neighbour resolution from the pin the pre-screen records when it admits a SYN — the frame's source MAC, held to the address's switch-derived MAC and dropped when the row withdraws — as one synthetic ARP reply fed back through the device's receive queue. A SYN from a registered address whose source MAC is not its switch-derived MAC is reset and never reaches a listener; a retransmitted SYN on a tuple the pool already holds is dropped, not handed to a sibling's listener; reconcile runs before the screen so a row that registers in the same turn as its first SYN is screened against the share that turn added.
…andle A boot under MINVMD_BEP_STUB exists to be probed through the stand-in, so a bind failure now fails the boot where the bind happens — the same argument the switch-spawn failure beside it already makes — instead of booting on without a listener for the lane to find. The pool's box source is now wired from whether the stand-in actually bound (_bep_stub.is_some()), not from whether the flag was set: the pool can never be partitioned by a path nothing is listening on.
The proxy-source case now asserts, from this case's daemon log, that each box registered with the VM host daemon before any probe runs, and fails through bep_fail naming the box that never did — an own_ip box with no row in the box-egress pool's partition is reset with nothing on the CLI's stderr naming the cause, which is the exact posture difference the case exists to catch. The peer-up gate no longer greps for the stand-in bind failure: a bind failure now fails the daemon's boot, so it surfaces at the activation above and never as a booted daemon the gate could catch. The macOS provider default's comment names #1816 as the fix that retires it.
…-accepted-box-connection-to-the-proxy
|
Round 8 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. At this branch the box egress proxy's pre-screen pins each registered box's address to the MAC its admitted SYN came from, resets (with an audited warn naming the source) a SYN whose source MAC is not the address's switch-derived MAC, drops the pin when its row withdraws, and answers the peer's own neighbour ask from the pin — one synthetic ARP reply fed back in through the device's receive queue, so the peer never puts an ARP request on the lane and an unpinned address is answered with nothing; a bare SYN on a tuple the pool already holds is dropped so the held socket answers it and a sibling's same-turn SYN still finds its listener; a row that registers in the same turn as its first SYN is screened against the share that turn's reconcile added; a stand-in bind failure under MINVMD_BEP_STUB fails the daemon's boot and the pool's box source is wired from whether the stand-in actually bound; and the proxy-source e2e case asserts both boxes' registration with the VM host daemon before any probe runs. The four switch lines and two minvmd lines pass at the head; the two VM boxes still arrive at the proxy's unix socket from their own switch addresses, a same-uid host process without the per-boot token is refused, a box past its cap is reset while its siblings keep their shares, and a stalled acceptor holds one box's flow while the others keep moving — with the macOS e2e and KVM test-kvm lanes owing the 'proxy sees each VM box by its switch address OK' line at the new head. Notes from the agent: One self-inflicted defect to know about: the minvmd commit's header runs 76 characters ('fix(minvmd): fail the boot on a stand-in bind failure, wire off the handle') where the working rule caps headers at 72 — the repo's own commitlint (config-conventional, header-max-length 100) accepts it, and a new commit cannot repair an old subject while rewriting one is forbidden, so it stands as written. Deferred: the e2e test line (KVM/libkrun lane) could not run on this box —…[truncated] |
|
/revise Round 9. Round 8's production changes are all in and read correct. Two of the tests it asked for pass even with their fix reverted, and the leg still learns neighbours from inbound ARP. Scope:
Unpinned addresses keep getting no answer at all; that silence is intended. Leave room in the pre-screen for a later per-port active reject: do not fold UDP handling into a single drop path that a reject cannot be added to. Evidence for this round:
The PR body is edited by hand; do not rewrite it. |
smoltcp fills the interface's neighbour cache from any ARP packet aimed at the leg — requests included — taking the sender's address and MAC as given, so a box could claim a sibling's address with its own MAC and re-point that sibling's return bytes. The pre-screen now rules every inbound ARP request or reply in its own arm: it passes only when the sender pair is a pinned pair or the address's switch-derived MAC, and drops it with a throttled warn naming the claimed address otherwise. Valid requests still pass, so a box that has never connected can still find the leg, and the leg still answers ARP for its own address. Unpinned addresses keep getting no answer at all; that silence is intended, and the pass-through arm stays separate so a later per-port active reject can be added to it. The retransmit test now puts a free sibling listener ahead of the held socket in set order — B first, then A; B's slot refreshed at the front — so the starve-the-sibling path is reachable. The reuse test now exercises the pin drop itself: the pin for a withdrawn address is gone (read through a new test_util accessor), a between-rows frame gets no delivery, and a re-registered occupant of the same address is delivered with every frame addressed to its derived MAC and no ARP on the lane at all. A new withdrawal test pins that a withdrawn row's flow is aborted and no frame for it leaves the peer afterwards, and reconcile's doc records why the sockets end before the pin drops: the neighbour cache outlives unpin by up to 60 s, safe only because no socket is left to send into the stale entry and the address-reuse quarantine outlives the cache.
…-accepted-box-connection-to-the-proxy
|
Round 9 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. Two VM boxes connecting to the proxy's address still arrive at its unix socket from their own switch addresses in the delivery header, and at this head a box can no longer borrow a sibling's address: the pre-screen drops every inbound ARP request or reply whose sender pair is neither a pinned pair nor the address's switch-derived MAC (throttled warn naming the claimed address), so B's return bytes reach only B at B's MAC; the retransmit test now holds a free sibling listener ahead of the held tuple and the reuse test now shows the pin itself dropping — a between-rows frame gets no delivery, and a re-registered occupant of the released address is delivered through its own new row with every frame addressed to its derived MAC and no ARP on the lane at all; and a withdrawn row's flow is aborted before its pin drops, so no frame for it leaves the peer afterwards, with reconcile's doc recording why that ordering is safe (the neighbour cache outlives unpin by up to 60 s; the address-reuse quarantine must stay longer). Notes from the agent: Scope held: only crates/switch/src/bep_host.rs changed, per the directive. Mutation evidence (each applied alone, then reverted; all reverts confirmed by a green bep_host run): (1) with the taken-tuple branch enqueueing instead of dropping, a_retransmit_on_a_held_tuple_takes_no_siblings_listener FAILS — 'the sibling's same-turn SYN was delivered, not starved by A's retransmit: ""' — because the sibling's same-turn SYN reaches the free listener when the retransmit is held out of the set; (2) with self.answers.unpin(*addr) removed from reconcile, a_reused_address_arrives_only_through_a_new_row FAILS — 'the pin went with the row: left Some(Address([82, 84, 0, 64, 0, 2])) right None' — the pin is observable through the new TestLane::pinned_mac accessor; (3) with the pre-screen's drop removed (claim.holds() || true), a_gratuitous_arp_cannot_re_point_a_boxs_return_bytes FAILS — 'the claim was answered with nothing: no reply, no f…[truncated] |
|
Reviewed: 2 findings, none blocking at Address the comments on the diff. Clobber: none: |
There was a problem hiding this comment.
Reviewed through r9 against the T69 plan block and the ARCH B2 rulings; minimal-review LOWs deferred to gominimal/inbox#805. KVM and macOS e2e print the switch-address OK line at 030cefd.
T69 — Deliver an accepted box connection to the proxy over its own socket with the box's identity
slice: S10
kind: task
covers: NET-132
size: L
est: 450 code, 300 tests
executor: agent
difficulty: needs care
model: strongest
diagnostics: the bundle's VM host daemon log tail carries each delivered connection with the box's switch address, port and the acceptor socket, each refusal with its reason, and the per-source socket count at each cap hit
observability: one debug log line per connection delivered with the source switch address and the acceptor socket; one rate-limited warn line per connection reset at the cap or for an acceptor that is down
layers:
- switch:
- crates/switch/src/bep_host.rs: behind the stack-peer feature; a pool of listening sockets at the proxy port, partitioned per registered box so each box's share is the per-source cap, added at registration and withdrawn with its row; an accepted socket's remote endpoint names the box; past the cap the socket is aborted; the connection is delivered by dialing the proxy's unix socket, one connection per flow and never reused, sending the per-boot token, then one fixed header (a leading u8 version, 1; a 16-byte box id, all-zero until T44 fills it; the box's switch address and source port; the destination address and port), then pumping bytes both ways under the socket's own window; an acceptor that is down aborts the socket
test: cargo nextest run -p switch --features stack-peer connections_past_the_per_source_cap_are_reset
test: cargo nextest run -p switch --features stack-peer one_boxs_exhaustion_never_starves_a_sibling
test: cargo nextest run -p switch --features stack-peer a_slow_acceptor_stalls_one_flow_not_the_stack
test: cargo nextest run -p switch --features stack-peer a_down_acceptor_resets_the_boxs_connection
- vmhost:
- crates/minvmd/src/net/bep_stub.rs (new): test and e2e only; the stand-in acceptor on a unix socket it owns at mode 0600; it takes the per-boot token at start, checks the peer's uid against its own and on Linux the peer's pid against the host daemon's, then the token, then the header version, and refuses and audits on any mismatch; it answers one line naming the source presented
- crates/minvmd/src/cmd/run.rs: mints the per-boot token, hands it to the peer in-process and to the stand-in over its start-up channel, and names the proxy socket path and the per-source cap
test: cargo nextest run -p minvmd delivered_connection_arrives_from_the_boxs_switch_address
test: cargo nextest run -p minvmd host_process_never_arrives_from_a_box_address
- docs:
- docs/specs/18-spec-box-networking/18-spec-box-networking.md: NET-132 records the per-source cap (16) and the per-box pool partition as proposed working values, tuned by the integration run
- e2e:
- scripts/session-e2e.sh: adds the case proxy_sees_each_vm_box_by_its_switch_address against the stand-in acceptor
test: ./scripts/session-e2e.sh proxy_sees_each_vm_box_by_its_switch_address
produces:
- crates/minvmd/src/net/bep_stub.rs
consumes:
- crates/switch/src/bep_host.rs
- crates/minvmd/src/box_registry.rs
proof: Two VM boxes connecting to the proxy's address arrive at its unix socket from their own switch addresses in the delivery header, a same-uid host process without the per-boot token is refused and never presented as a box, a box past its cap is reset while its siblings keep their shares, and a stalled acceptor holds one box's flow while the others keep moving.
depends on:
- blocked by #1627
- blocked by #1652
What changes
switch,stack-peer). The host leg holds a pool of listening sockets at the proxy port (8118), partitioned per registered box. Each box's share is the per-source cap (16, a working value). A share is added at registration and withdrawn with its row.(source, port)the pool already holds is dropped (the held socket answers its own retransmits), so it can never take a sibling's listener.enforce_capsstays as the second line of defence.NoBoxes, so it binds no pool socket until the credentialed-only gate rule (T45, T45 — Reach the proxy only as a credentialed lane's infrastructure #1665) lands. The pool is partitioned by the box table only when the e2e stand-in actually bound (MINVMD_BEP_STUB). Under that flag, a stand-in bind failure or a switch that will not come up fails the boot.minvmd, test/e2e only).SO_PEERCREDon Linux,getpeereidelsewhere) and, on Linux, the peer's pid. Then it checks the token (constant-time) and the header version.proxy_sees_each_vm_box_by_its_switch_addressactivates two own-IP VM boxes.--provider local-minvmdso that boxes register with the host table. fix(minimal): register own-ip boxes on macOS without an explicit --provider #1816 retires that default.Tests
cargo nextest run -p switch --features stack-peer connections_past_the_per_source_cap_are_resetcargo nextest run -p switch --features stack-peer one_boxs_exhaustion_never_starves_a_siblingcargo nextest run -p switch --features stack-peer a_slow_acceptor_stalls_one_flow_not_the_stackcargo nextest run -p switch --features stack-peer a_down_acceptor_resets_the_boxs_connectioncargo nextest run -p minvmd delivered_connection_arrives_from_the_boxs_switch_addresscargo nextest run -p minvmd host_process_never_arrives_from_a_box_address./scripts/session-e2e.sh proxy_sees_each_vm_box_by_its_switch_addresse2eOK (run 37071710474), KVMtest-kvmOK (job 111060965361). At 030cefd: fill in macOSe2eand KVMtest-kvm. A local run SKIPs (no switch).Also covered:
an_expired_neighbour_is_answered_from_the_pin_never_arpa_syn_with_a_foreign_mac_is_reset_and_never_delivereda_reused_address_arrives_only_through_a_new_rowa_retransmit_on_a_held_tuple_takes_no_siblings_listenera_gratuitous_arp_cannot_re_point_a_boxs_return_bytesa_withdrawn_rows_flow_sends_no_frame_after_its_reseta_row_registered_with_its_first_syn_takes_no_sibling_listenerproduction_wiring_leaves_pool_len_zero_with_rows_registeredOutside the task's layers
crates/minvmd/src/net.rs: theHostGvproxy::spawn/BepPeer::spawnarity, the test call sites, and thebep_stub/BepWire/BepBoxSourcere-exports.crates/minvmd/Cargo.toml: aswitchdev-dependency withtest-util, for the delivery tests.crates/switch/Cargo.toml: declares thetest-utilfeature.Closes #1794