Repository navigation
T63 — Gate the switch's publish verbs on the host and hand the node its ports - #1788
norrietaylor merged 42 commits into
Conversation
The frame half of NET-081's gate decides what a box may send out; this adds the other half, the decision over what its daemon may ask the switch to publish. applied() is pure over an owned request summary (verb, switch address, up to four records as ports or name indices), the host-side table NET-138 fills, and the unregistered-source phase: a publish is applied only where a row holds the address and admits every record; a retraction is decided table-wide, by what the rows hold rather than where the guest sits; an address no row holds is the phase's to decide, mirroring the frame verdict's lease-block interim. The Kani harness pins the decision as an iff over a fully symbolic request and row, in the frame verdict harness's shape.
… ran out the coder's uncommitted work as it stood when its 7200s ran out The coder was stopped at the end of its time with these changes not committed. The runtime committed its working tree as it stood so the work is kept: this commit is a snapshot, not a finished step, and may not build or pass its tests.
…witch-s-publish-verbs-on-the-host-and
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
/revise Round 1 on the out-of-time snapshot (db01a5a). The snapshot carries the core and vmhost layers only; the daemon layer in the plan block is untouched and two of the five plan test lines do not pass. Bound for this round:
Nothing else is in scope for this round. |
The drainer took the report channel's receiver from a clone, whose withdrawal_reports_rx is always None by the Clone contract — so no drainer thread ever spawned and no row was ever withdrawn; a box's row outlived its shuttle connection past the box-end bound (NET-133). Take the receiver from the registry that created the channel.
The node-port assignment helpers compile everywhere but are only called from run_foreground, which is libkrun-gated — so every libkrun-less build of the lib warned them dead. Allow them under not(minvmd_libkrun), the pattern DetachPoll and vm.rs's helpers already carry; the tests still cover them on every target.
clippy asks a # Errors section of a fn returning Result; applied's refusals (UnknownAddress, Undeclared, Unheld) get theirs.
minvmd assigns the node's hostname-proxy (TCP) and zone-answerer (UDP) ports before the VM boots and hands them on the kernel command line; pid-1 reads the tokens off the environment the kernel passes through and binds them as handed: pinned, never re-picked, never relocated. The host's box table already names exactly these ports, so a daemon-chosen replacement would publish a listener the host gate refuses to carry. A boot without tokens — an older minvmd, a native run — keeps the pre-handoff default-then-select behaviour.
The exhaustive decision test was 177 lines, over clippy's too-many-lines ceiling, and summed its width range the pedantic lints reject. The sections become named helpers the survey-named test still drives in the same order; no case changes.
Three control-relay tests failed against the gate they shipped with: - the pipelined-request and connect-path-body tests read the relayed bytes through gate_over_control, which already consumes and asserts the whole admitted request — the body re-read found the head's bytes under it, and the pipelined test asserted the smuggled request was forwarded. Both now write by hand and read exactly the bytes the gate owes the switch. - a switch hanging up on an idle control connection ended only the response leg; the relay_control implementation never raced its legs, so the per-connection close a keep-alive control channel makes went unseen until the guest spoke again or the drain bound passed. Race the response leg against the request-leg probe like the frame relay does, and say which leg ended.
…witch-s-publish-verbs-on-the-host-and
|
/revise Round 2. Round 1's work was recovered from the quarantine ref and main was merged in by hand at dfcbdff (the egress-gate conflict with T37's node-plane baseline is resolved;
Nothing else is in scope for this round. |
The test held its busy port on 127.0.0.1 while assign_node_port probes 0.0.0.0, a collision Linux raises and other hosts do not, so the busy case's fall-through was not asserted everywhere. Holding each probe on the wildcard address the assigner itself addresses makes the collision unconditional, and the freed case the same inverse of the busy one.
|
Round 2 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. On a VM-backed host the in-VM daemon's request to publish a port or a name at the switch is applied only when the host-side table holds its address and admits the port or name: the gate reads each control request on the shuttle socket, decides it against the registry's row for the request's address, forwards an admitted request verbatim, and refuses the rest with a rate-limited warn line naming the rule, the address, the port or name, and the reason; the node's hostname-proxy and answerer ports are assigned by the VM host daemon before the VM boots, handed to the guest on the kernel command line, read by the guest daemon, and bound by it in pid-1 mode as handed. All five plan test lines pass at the head of this round, including the withdrawal line that exited 100 at the round-1 snapshot. Notes from the agent: Revise round 2, bounded to the directive's three items. (1) host_table_row_withdrawn_within_60s_of_box_end passes at the new head: no change was needed. Its body is byte-identical to the snapshot's and the merge resolution is the only change to the code it exercises since (GateAdmit::Baseline attribution in relay_frames_to_switch; the test's row admits through GateAdmit::Row either way), so the snapshot's exit 100 does not reproduce here — it passed 8 consecutive runs plus the recorded one at 4aedc7d. (2) node_port_assigned_on_host_and_handed_to_daemon now holds its busy and freed TCP ports and its busy UDP port on the wildcard address assign_node_port probes (0.0.0.0), so the held-vs-probe collision is unconditional on every host instead of Linux-only; committed as 4aedc7d, the only change of the round. (3) All five plan test lines re-run at that head, exit 0 each; the Tests table and the proof replacing the 'Out of time' section are carried by this file for the body refresh, and un-drafting on agent-verify is the runtime's action. Deferred: cargo nextest run -p sessions kani_switch_request_applied_iff_table_admits (lane: kani) — the harness exi…[truncated] |
…olds The publish-verb proof restated the announced interim as any in-plan address, so a publish at a held address whose row refuses a record read as applied while the decision refuses it under either phase. The oracle now grants the interim only where no row holds the address, and a retraction's interim only where no row holds its records. The Kani lane expects the ninth sessions harness.
…witch-s-publish-verbs-on-the-host-and
The merge of main brings #1792's vm_escape test, which registers the node namespace with the signature main had; this branch's takes the node's proxy and answerer ports.
|
/revise Design review (ARCH, against deployment-and-egress-gateway.md v0.9 §4.2, §4.3, §5.5, §7.1) at head 26ca687. Lanes are green; these are design corrections, every one bounded to the plan's layers.
Re-run the plan's test lines after the changes:
Do not run |
…witch-s-publish-verbs-on-the-host-and
A retraction was decided table-wide: it applied whenever any published row held the record it names, so a retraction could withdraw a record a publication at another address had made. Read the row at the address the summary carries — the publication's own, as the gate attributes it — and apply only when that row holds every record; otherwise Refusal::Unheld, which now carries the address for the refusal line to name. The announced interim stays for an in-plan address no row holds, as for a publish. The Kani harness restates the admit per address for every verb, and the unit test pins a retraction justified only by another namespace's row as refused.
The unexpose body names only the loopback listener it retracts, and the pure decision now applies a retract by the row at its own address — so the gate supplies that address itself, from a publish ledger kept beside its limiter: every applied publish files its listener and the address it was applied at, and an applied retraction takes its listener out. A retraction for a listener no applied publish names is keyed at the unspecified address, which no row holds, and refused under either phase; the same request was applied table-wide before, so a fabricated retract of a port some row declared travelled to the switch. The interim's own teardowns key at the lease their publish went out at, under the announced phase that applied them. The interim's admit line is marked interim and reads verb-neutral, and the phase's docs now say the two bounds its admits carry: read once per build, never changed while a VM runs; lease-run addresses only, never the node's own address or the plan's reserved pair. The withdrawal test is renamed for the connection it proves — the box's shuttle relay, which ends with the box's traffic, not with the box: creator-driven withdrawal at destroy lands with T66 (#1711) — and the registry drainer's doc says the same.
A boot token present but unusable — unparseable, out of range — was warned about and counted as absent, sending the daemon into its own default-then-select and publishing listeners the host's box table does not name. The read is now a Result: only an absent token, or the host's own 0 as 'pick one yourself', leaves the pre-handoff fallback; anything else is a surfaced boot error naming the token and the value it carried. The pid-1 branch reads the tokens once the log sink is live and probes the handed pair in the microVM's bind base before the daemon starts, so a port something in the guest already holds ends the boot with the error surfaced, never after READY when the host already believes the VM healthy and never rescued by a re-pick.
Directive finding 4 (T63 round 5): run.rs resolved the pair itself while vm.rs re-read MINVMD_NODE_*_PORT off its own env and silently fell back to absent on garbage, so a half-set or mistyped override could boot the guest onto ports different from the ones the node row registered. Now run.rs resolves each port once — the operator's override in the supervisor's own env (the same names the child transport decodes), refused fail-closed when it is unparseable or 0, else the default-first probe — and the one resolved pair is written twice from the same value: onto the VMM child's env (the explicit .env shadows any inherited override) and into the node row's own publishes. vm.rs's decode becomes strict transport decoding: both-or-neither, a half-pair or an undecodable value fails the boot naming the variable and the value. Its read stays in vm.rs because the decode's only production caller is apply(); moving it to cmd/vmm_child.rs is out of this task's layers. node_port_assigned_on_host_and_handed_to_daemon (both the run.rs and the vm.rs test of that name) now pins the resolution through the override and asserts handed == registered, plus the decode's strict cases.
|
Round 5 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. On a VM-backed host the in-VM daemon's publish, name-publish and retract requests at the switch are applied only when the host-side table holds the request's own address and admits its records — a retract is keyed at the address the publication it retracts was applied at, and an in-plan publish no row holds is an interim admit marked interim on its rate-limited warn line — every other request is refused with the same class of warn line naming the address, the port or name and the reason; and the node's hostname-proxy and zone-answerer ports are resolved once by the VM host daemon (operator override or default-first selection), registered as the node row's own publishes, and bound by the guest daemon exactly as handed, with a broken handoff or an unbindable handed port a surfaced boot error. Notes from the agent: All five directive items addressed, each in its own commit on top of the round's branch (96515f1 retract address-keying, 722f30e gate attribution + interim flag + test rename + Display reword, cd679d3 fail-closed handed ports, 8c4fcba one node-port resolution). Deferred: the kani lane line (cargo nextest run -p sessions kani_switch_request_applied_iff_table_admits) was not run — the lane proves it — and no harness was added (the existing per-address harness was extended in an earlier round), so scripts/kani.sh's 9-harness expectation needs no bump. Layer bound on item 4: vm.rs still reads MINVMD_NODE_*_PORT because the decode's only production caller is apply(), reached through cmd/vmm_child.rs, which is outside this task's layers; the read is now a strict fail-closed transport decode (both-or-neither, undecodable value fails the boot naming the variable and the value), and moving the read into the VMM child is the follow-up. crates/minvmd/src/error.rs (outside the layers) has no dynamic-message variant, so those decode errors ride VmError::Io with io::Error::other messages that name the variable and the value. The operator override shares t…[truncated] |
The gate's loopback_port parsed only 127.0.0.1:<port>, so every own-address box's ingress expose - and its teardown - was refused as malformed before the row decision: on a VM-backed host the daemon's client builds local from the box's granted address in the reserved local range (NET-010, NET-129), never 127.0.0.1. Accept both spellings the client builds - 127.0.0.1 and the reserved local range - and keep refusing every other spelling as malformed, with a test on each side of the line.
…witch-s-publish-verbs-on-the-host-and
|
Round 1 of 10 on the runtime's review findings, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. Review threads the coder judged:
On a VM-backed host the in-VM daemon's request to publish a port or a name at the switch is decided by the host-side table alone: the gate now summarizes the box's own leased-loopback spelling of Notes from the agent: Revise round 1: the one review thread (egress_gate.rs:2004, loopback_port accepting only 127.0.0.1) is fixed in commit 76a1b65 — loopback_port now parses |
…drop The gate defers one drop class for a row that declared DNS hosts: the undeclared destination, decided in the guest where the resolution-time pins are. Pin the edge of that deferral from the other side: for the same row shape, a destination in its own deny_subnets, the switch's own address, a protocol its rules do not allow, and a source no row holds are all still dropped on the host, under their own rules, with no undeclared-destination line. Say so at registration: BoxRegistry::register emits one info line per name-declaring row, naming its switch address, so the host's log carries the deferral rather than leaving it to be inferred from a frame that reached the switch. The line goes away with the deferral; moving the decision host-side is #1808.
The drainer's doc comment claimed the thread runs until the reports' senders are all gone — the gate and every table handed out. That exit is unreachable: the thread holds a clone of the registry, and a clone carries a withdrawal_reports Sender — the very channel the thread is draining — so recv() never reports the channel dead and the loop never exits. Say the process-lifetime truth instead, and say outright that teardown cannot rely on the thread exiting, so no future reader builds teardown on an exit that cannot happen.
|
Round 2 of 10 on the runtime's review findings, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. Review threads the coder judged:
On a VM-backed host the in-VM daemon's request to publish a port or a name at the switch is applied only when the host-side table holds its address and admits the port or name, every other request is refused and logged, and the node's hostname-proxy port is assigned by the VM host daemon at boot and bound by the guest daemon as handed — and the withdrawal drainer's documentation now states truthfully that the drainer runs for the process's lifetime because its own registry clone keeps the report channel's sender alive, so no reader can build teardown on an exit that cannot happen. Notes from the agent: Revise round 2: the delta was comment-only — the spawn_withdrawal_drainer doc comment in crates/minvmd/src/box_registry.rs no longer claims the thread exits when the reports' senders are all gone. Deferred (lane: kani): cargo nextest run -p sessions kani_switch_request_applied_iff_table_admits — the harness it names exists at crates/sessions/src/core/switch_request.rs:880 but this box lacks the Kani lane. cargo clippy -p minvmd --all-targets and cargo fmt -p minvmd -- --check were clean; just clippy-strict was not run this round (local-only gate, no CI lane runs it, and the changed lines are doc-comment only). |
… every row The design's infrastructure deny set (section 5.3) applies to every box-plane packet, CIDR-admitted direct-IP flows included, so the host-side gate now decides it as a frame rule for every row, between the source's routing and the row's own rules: link-local and the metadata services in it, loopback, the fabric plane outside the node's own switch block, and RFC 1918 space the row's allow_subnets does not cover. A row that declared DNS hosts defers only its undeclared-destination drop; this one is the host's own, whatever the row. The switch's gateway stays with the control-surface rule, and the node's own block is local reach the row's CIDR rules and the target's ingress decide. The host rule is built beside the sessions crate's InfrastructureDenySet rather than from it: that set keeps its ranges private, names the gateway's addresses, and admits or refuses the whole plane by whether a name is a box-zone name, which the rebinding intersection relies on and a frame cannot answer. An undeclared allow_subnets counts as covering private space, as the rebinding exemption holds it, so the two allow-all spellings keep the same reach. The drop is egress-infrastructure-destination, rate-limited per source like the other drops and naming the destination and port. The spoof-union test's db box moves from 192.168.0.0/16 to TEST-NET-1 so its own subnet is not private space, and its two spoofs into web's private subnet are pinned to the new rule.
A source the announced interim admits without a row returned before the infrastructure check, so its frames reached link-local and other nodes' blocks unrefused. The set applies to every box-plane packet: the interim now takes it too, as allow-all for RFC 1918 space, and the infrastructure test covers such a source.
A forwarder listener is its loopback address and port. Two boxes publishing one port at their own reserved-range leases are two listeners, so the ledger now keys each publish and retraction by the pair the body's local names.
There was a problem hiding this comment.
Approved at a84036b. The host gate covers the publish verbs, the infrastructure rule applies to every row including interim-admitted sources, and the publish ledger keys a listener by loopback address and port. ARCH's conditions are met, and the name-resolving row's deferral is tracked in #1808. Lanes are green and minimal-review approved with no findings.
The merge from main brought T63 (#1788), whose register_node_namespace now takes the node's proxy and answerer ports as its admitted pair. bep_stub.rs still called the zero-argument form, so the minvmd lib test target — both of this task's test lines and the agent-verify gate — failed to compile at all. The test now registers the node row the way cmd/run.rs does at VM boot, with the default pair (7654, 7656) the assignment resolves when no operator override pins it — the same literals every sibling test in the crate uses.
…the box's identity (gominimal#1810) * feat(switch): the coder's uncommitted work as it stood when its 7200s ran out the coder's uncommitted work as it stood when its 7200s ran out The coder was stopped at the end of its time with these changes not committed. The runtime committed its working tree as it stood so the work is kept: this commit is a snapshot, not a finished step, and may not build or pass its tests. * fix(switch): the bep_host pool tests' fixtures must carry real checksums The fixtures built TCP and UDP segments with ChecksumCapabilities::ignored(), leaving the TCP checksum zero. The peer's stack now verifies inbound TCP checksums at parse (BepDevice::capabilities advertises Both), so those segments were dropped before the stack could answer them: three T43-era tests expected a reset or a port-unreachable that never came. A real box kernel never sends a zero TCP checksum, so the fixtures now emit with the default capabilities, which fill every checksum. peer_binds_no_socket also counted the ARP reply among the answers its poll produces, but arp_exchange — which the pool rewrite added to the test — consumes that reply while teaching the stack its neighbour. The count is back to the two answers the poll actually produces: the reset and the port-unreachable. * feat(vmhost): wire the proxy's delivery and its stand-in acceptor The pool in switch::bep_host delivers a box's connection by dialing the proxy's unix socket, token first, then the fixed header naming the box's own switch address. The supervisor now carries the other end of that wiring: it mints the per-boot token beside the marker nonce, derives the proxy socket path beside the switch and gate sockets, names the recorded per-source cap, and hands the peer its wire in-process. Until the proxy's own acceptor lands, a stand-in accepts in its place: crates/minvmd/src/net/bep_stub.rs, a test/e2e surface the supervisor binds only when MINVMD_BEP_STUB is set — the e2e lane's flag, nothing else's. It takes the same token over its start-up channel, checks every connection in order — the peer's uid against the daemon's own, then on Linux the peer's pid against the host daemon's (SO_PEERCRED, read before any presentation byte), then the token, then the header version — and refuses and audits on any mismatch, closing without an answer so nothing refused is ever presented as a box. A connection that passes is answered with one line naming the source it was presented from. The registry's rows become the pool's partition through RegisteredBoxes over the table's live view, so a row that lands grows its box's share within a stack turn. HostGvproxy::spawn threads the wire and the box source to the peer beside the switch runtime; its tests pass a socketless source and an acceptor path nothing listens at, which is the acceptor-down answer the pool gives on a production boot. * docs(net): record NET-132's proposed working values Annotate the requirement with the working values the switch and the host daemon ship: the proxy's listener on port 8118, a per-source cap of 16 concurrent connections from one box, and the delivery's listener pool partitioned per registered box so each box's share is that cap, added at its registration and withdrawn with its row. All proposed working values, to be tuned by the integration run. * test(e2e): read each box's delivery source at the stand-in acceptor Add the NET-132 case proxy_sees_each_vm_box_by_its_switch_address: activate two boxes, connect each to the proxy's address through the switch, and read the delivery's source at the stand-in acceptor the host daemon mints a per-boot token for when MINVMD_BEP_STUB is set. Each box's answer must name its own switch address and the proxy's endpoint, and neither box's answer may name the other's address. The case is gated on the switch (MINVMD_GVPROXY_BIN) like its T43 neighbours, so the no-switch native lane skips it, and it runs last in the whole-lane order because it restarts the daemon to hand the stub its token. * style(vmhost): apply rustfmt to the box egress delivery files Straighten the import order in bep_stub.rs and wrap the RegisteredBoxes projection so both files pass the fmt-check gate the linux native lane runs. * fix(vmhost): keep the stand-in's pid proof on Linux The macOS unit lane runs `cargo clippy -p minvmd --all-targets -- -D warnings` and then the minvmd suite, and both failed there, for two reasons one cfg short of each other: - `AsRawFd` was imported unconditionally but used only inside the `#[cfg(target_os = "linux")]` peer-credentials read, so off Linux the import read as unused and `-D warnings` turned it into an error. - `host_process_never_arrives_from_a_box_address` asserted the pid refusal end to end, and only `SO_PEERCRED` can prove that: off Linux `peer_credentials` names no pid, so the foreign-process presentation was answered rather than refused, and the answer-empty assertion fell. Gate both to Linux — the layer binds the pid check to Linux in the same breath — and reword the test's doc comment so the pid leg is named as Linux's. The uid check, the per-boot token and the header version are portable refusals and stay unconditional. Verified by inverting the file's cfgs and re-running clippy `-D warnings` and both delivery tests under the non-Linux semantics, then restoring them. * fix(e2e): the proxy-source probe discarded the answer it asserts on The NET-132 case handed socat /dev/null as its left address — the shape the MAC and ARP cases use, which read only socat's stderr and the box's ARP table — so the stand-in's answer line went to /dev/null and the case's greps always found nothing. Give the probe an empty pipe as socat's left address with -t 10, the shape the hostname-proxy case's bogus-head probe uses, so the delivered line lands on the box's stdout. Also take the case's two seed dirs into the EXIT trap, since every failure path in it goes straight to fail. * fix(e2e): guard the proxy-source seed dirs against set -u The teardown's new rm -rf lines named two variables nothing assigns until the case runs, and the script runs under set -u: every run aborted the EXIT trap with 'unbound variable' and skipped the workdir cleanup, even on paths the case never touches. Initialise both to empty beside the lane's other seed dirs, so the trap sees them on every path and the failure paths' dirs are the only ones it removes. * fix(switch): deliver a box connection that half-closed at connect smoltcp folds a FIN that arrives with the handshake-completing ACK straight from SynReceived into CloseWait, and the daemon's turns are a hundred milliseconds apart, so a box that finished sending as it connected - the e2e probe's shape - always reached the service pass as CloseWait. The pool only dialled Established sockets, so that connection hung: never delivered, never reset, its slot lost to the share. Deliver CloseWait accepted connections too. The FIN only closed the box's write side: smoltcp still lets the socket transmit, so the acceptor's answer still reaches a box that can only listen. Pinned by a_half_closed_connection_is_still_delivered, which queues the box's FIN behind its handshake ACK between turns; without the fix the acceptor takes nothing. * fix(minvmd): pass the node port pair the merged T63 signature requires The merge from main brought T63 (gominimal#1788), whose register_node_namespace now takes the node's proxy and answerer ports as its admitted pair. bep_stub.rs still called the zero-argument form, so the minvmd lib test target — both of this task's test lines and the agent-verify gate — failed to compile at all. The test now registers the node row the way cmd/run.rs does at VM boot, with the default pair (7654, 7656) the assignment resolves when no operator override pins it — the same literals every sibling test in the crate uses. * fix(switch): hold a box's share before the stack assigns a listener The pool's slots are anonymous, so smoltcp assigned inbound SYNs to any listening socket in arrival order: a box that opened more connections than its share in one turn took a sibling's listener, and the sibling's same-turn SYN met no socket at all and was reset. BepStack::poll now pre-screens the inbound frames before the interface sees them and refuses a connection-opening SYN to the proxy port whose source has no row or already holds its per-source cap, answering each refusal with a reset the box's own stack accepts and one rate-limited warn naming the source and the reason. A retransmitted SYN of a connection the pool already carries passes through, so the cap never counts one connection twice. one_boxs_exhaustion_never_starves_a_sibling now drives the burst it always meant to: box A opens cap + 2 connections and box B opens one before a single turn, and B is still delivered. The peer also never emits an Ethernet broadcast again: an outbound ARP request's destination is rewritten to the switch-derived MAC of the address it asks about, so a 61-second neighbour expiry re-resolves over the point-to-point lane instead of reaching every box on the plan, and a_re_resolved_neighbour_is_aimed_not_broadcast pins it. pump_to_acceptor now notifies the stack after each successful write, so a drained channel reopens the box's window without waiting for the 100 ms tick. The module docs carry the gate sentence again — T69 is the only task that may bind a socket on the peer, and until the credentialed-only gate rule (gominimal#1665) lands, the supervisor hands the peer the new empty source, NoBoxes. * fix(minvmd): bind no pool socket on a production boot The supervisor hands HostGvproxy::spawn the new empty source unless the stand-in is enabled: a socket in the pool is a lane onto the host for the box whose share holds it, and the credentialed-only gate rule (gominimal#1665) is still ahead, so rows registered or not — the guest node namespace's row included — a production boot partitions nothing and the pool binds nothing, leaving a box's connection to the proxy's address to the stack's own reset. Only MINVMD_BEP_STUB, the e2e lane's flag, wires the table's box rows. production_wiring_leaves_pool_len_zero_ with_rows_registered pins both wirings: zero with rows registered, the per-source cap per box row when the stand-in asked for them. The node namespace's row is the guest's own root netns, the daemon's tap — never a box — so RegisteredBoxes excludes the subnet's daemon address from the rows it hands the pool: it sits outside the hand-out run every client box is allocated from, so excluding that one address names exactly the node row. delivered_connection_arrives_from_the_boxs_ switch_address now asserts the node row buys no share and that a connection from the daemon address is reset and presented as nothing. Off Linux the stand-in reads the peer's uid with getpeereid instead of handing back its own, so the uid check is real on macOS and host_process_never_arrives_from_a_box_address covers it; no pid comes with it, so the pid check stays the Linux extra. The per-boot token is compared in constant time: a byte-at-a-time compare would leak how many leading bytes a guess got right through when the refusal closes the connection. test_util re-exports smoltcp's tcp State so a test in another crate can name it. * fix(e2e): stop the host daemon before the stand-in's flag goes out The proxy-source case stops only the session daemon before exporting MINVMD_BEP_STUB, so on a VM lane the MAC and ARP cases above leave a minvmd running with no stub flag, the activation finds it, and the whole case runs against a daemon that bound no stand-in and no pool: the boxes' connections to the proxy's address were reset and the case died in a blank answer that named no cause. The case now stops both, the pair teardown stops — min stop, and minvmd stop on a VM lane — before the flag goes out, so the activation autospawns the one daemon that carries it. The case also asserts the stand-in's socket exists before any box probes: the daemon binds it at boot only under the flag, at the path beside the switch socket, so a daemon that came up without it fails there, with the cause named. Every failure path now prints the host daemon's own log tail first — the autospawned minvmd writes a daily `minvmd.log.<date>` whose dated name the global fail()'s `*.log` pattern never reaches — through bep_fail, which drops the flag, names that log, then the global diagnostics. * fix(minvmd): refuse a switchless boot under the stand-in's flag A daemon booted under MINVMD_BEP_STUB exists to be probed through the box egress proxy's switch, and the stand-in's socket is bound before that switch is spawned — so a degraded boot looks healthy to anything that waits on the socket, and every probe below it fails in terms of a peer that was never there. The macOS lane's e2e run showed exactly that shape: activations healthy, socket present, the box's connection to the proxy's address refusing, with nothing in the annotation stream naming the leg that failed. Fail the boot when the switch will not come up while the flag is set (the e2e lane's and nothing else's — production boots keep the warn-and-degrade behaviour), so the cause surfaces at the activation that triggered it instead of three steps later. * fix(e2e): name the failed leg when the proxy-source probe does not complete The macOS lane failed this case twice on a healthy-looking boot: the activations succeeded, the stand-in's socket existed, and the box's connection to the proxy's address still did not complete — with socat's stderr and the daemon's log tail printed as plain text, none of which reaches the run's annotation stream, so the only thing the lane said from outside was the probe's own failure. Three moves, all inside the case: - gate the probes on the daemon's own log line for the peer, scoped to the lines this case's daemon wrote (the host log is shared by every daemon of the run; an unscoped grep reads the earlier cases' daemons' identical switch-up lines), so a boot whose switch never came up fails here naming the spawn, not at a box's blank answer; - when a probe fails, emit the deciding facts as annotations: the probe's stderr, the box's ARP entry for the proxy address, a same-box cross-probe to an unlistened port (its RST proves the box-to-peer path and isolates the failure to the pool's listener or the delivery's dial), and the daemon's own box-egress-proxy lines; - the answer-grep failures carry the same daemon-log annotation. Pair commit: the minvmd change makes the switchless boot these guard against fail loudly at boot. * fix(e2e): let the proxy-source case's daemon log what it asserts on The lane exports RUST_LOG=warn,minimald::exec=info for the whole script, and the minvmd this case autospawns builds its EnvFilter from that value, which drops every INFO record from minvmd's own modules — so the gate's 'host gvproxy switch up; box egress proxy peer started' line could never appear and the case timed out and failed on every lane that runs it. Widen the filter for this case's daemon only, restoring it on every exit, and hold the gate on both records: the stand-in's flag-gated line proves MINVMD_BEP_STUB reached this daemon, not just that the socket file exists, and the peer's line proves the switch came up behind it. The widened value also lets the pool's per-delivered-connection DEBUG line through, so the log tail a failure prints names the surface the case observes. * fix(e2e): default the provider args on the macOS VM lane The macOS lane runs the whole session e2e without E2E_MINIMAL_ARGS, so every mnl call reaches min without --provider local-minvmd. On macOS the CLI resolves the minvmd backend either way (client_provider_kind gates on the OS), so all earlier cases pass — but GlobalArgs::use_minvmd() is a strict flag check, and it gates the box registration and its withdrawal against the VM host daemon (T66). Without the flag the proxy-source case's own_ip boxes activate unregistered: the guest self-allocates its switch address from the reserve (100.64.0.2 in the failed run), the box-egress pool has no row to partition a share from, and the pool's prescreen resets the box's connection to the proxy's address — 'reset a connection from a source with no registered row' — with nothing on the CLI's stderr naming the cause. The KVM lane passes the flag and is green through the whole suite, so the script now supplies the same args for its one VM lane that cannot: an explicit E2E_MINIMAL_ARGS is kept verbatim and Linux is untouched, so the native lane keeps its backend. * fix(switch): pin the box egress proxy's neighbours, never ask the lane The neighbour architecture ruling (review): the peer answers its own stack's neighbour resolution from the pin the pre-screen records when it admits a SYN — the frame's source MAC, held to the address's switch-derived MAC and dropped when the row withdraws — as one synthetic ARP reply fed back through the device's receive queue. A SYN from a registered address whose source MAC is not its switch-derived MAC is reset and never reaches a listener; a retransmitted SYN on a tuple the pool already holds is dropped, not handed to a sibling's listener; reconcile runs before the screen so a row that registers in the same turn as its first SYN is screened against the share that turn added. * fix(minvmd): fail the boot on a stand-in bind failure, wire off the handle A boot under MINVMD_BEP_STUB exists to be probed through the stand-in, so a bind failure now fails the boot where the bind happens — the same argument the switch-spawn failure beside it already makes — instead of booting on without a listener for the lane to find. The pool's box source is now wired from whether the stand-in actually bound (_bep_stub.is_some()), not from whether the flag was set: the pool can never be partitioned by a path nothing is listening on. * fix(e2e): assert both boxes registered before the proxy-source probes The proxy-source case now asserts, from this case's daemon log, that each box registered with the VM host daemon before any probe runs, and fails through bep_fail naming the box that never did — an own_ip box with no row in the box-egress pool's partition is reset with nothing on the CLI's stderr naming the cause, which is the exact posture difference the case exists to catch. The peer-up gate no longer greps for the stand-in bind failure: a bind failure now fails the daemon's boot, so it surfaces at the activation above and never as a booted daemon the gate could catch. The macOS provider default's comment names gominimal#1816 as the fix that retires it. * style(switch): apply rustfmt to the neighbour-pin changes * fix(switch): drop unpinned ARP claims in the box egress pre-screen smoltcp fills the interface's neighbour cache from any ARP packet aimed at the leg — requests included — taking the sender's address and MAC as given, so a box could claim a sibling's address with its own MAC and re-point that sibling's return bytes. The pre-screen now rules every inbound ARP request or reply in its own arm: it passes only when the sender pair is a pinned pair or the address's switch-derived MAC, and drops it with a throttled warn naming the claimed address otherwise. Valid requests still pass, so a box that has never connected can still find the leg, and the leg still answers ARP for its own address. Unpinned addresses keep getting no answer at all; that silence is intended, and the pass-through arm stays separate so a later per-port active reject can be added to it. The retransmit test now puts a free sibling listener ahead of the held socket in set order — B first, then A; B's slot refreshed at the front — so the starve-the-sibling path is reachable. The reuse test now exercises the pin drop itself: the pin for a withdrawn address is gone (read through a new test_util accessor), a between-rows frame gets no delivery, and a re-registered occupant of the same address is delivered with every frame addressed to its derived MAC and no ARP on the lane at all. A new withdrawal test pins that a withdrawn row's flow is aborted and no frame for it leaves the peer afterwards, and reconcile's doc records why the sockets end before the pin drops: the neighbour cache outlives unpin by up to 60 s, safe only because no socket is left to send into the stale entry and the address-reuse quarantine outlives the cache. --------- Co-authored-by: minimal-foundry[bot] <327719912+minimal-foundry[bot]@users.noreply.github.com>
T63 — Gate the switch's publish verbs on the host and hand the node its ports
slice: S9
kind: task
covers: NET-081, NET-138, NET-025
size: L
est: 650 code, 250 tests
executor: agent
difficulty: needs care
model: strongest
diagnostics: the bundle's VM host daemon log tail carries each refused switch request with its address, port or name and reason
observability: one rate-limited warn log line from the VM host daemon per refused request class with the address, the port or name and the reason
layers:
- core:
- crates/sessions/src/core/switch_request.rs (new): the pure request-vs-table decision over an owned request summary (verb, switch address, up to 4 records as ports or name indices) and an owned table, with its Kani harness
- crates/sessions/src/core/mod.rs: declares the module
test: cargo nextest run -p sessions switch_request_applied_only_when_table_admits
test (deferred: kani): cargo nextest run -p sessions kani_switch_request_applied_iff_table_admits
- vmhost:
- crates/minvmd/src/net/egress_gate.rs: parses each control request on the shuttle socket into a summary, applies the decision against the table, forwards an admitted request to the switch and refuses the rest with one rate-limited warn line naming the address, the port or name and the reason
- crates/minvmd/src/box_registry.rs: the node's row with the helper-assigned proxy port; a box's row is withdrawn when its shuttle connection closes or its creator destroys it
- crates/minvmd/src/cmd/run.rs: assigns the proxy port before the VM boots
- crates/minvmd/src/vm.rs: hands the port to the guest on the kernel command line
test: cargo nextest run -p minvmd switch_request_refused_and_logged
test: cargo nextest run -p minvmd node_port_assigned_on_host_and_handed_to_daemon
test: cargo nextest run -p minvmd host_table_row_withdrawn_within_60s_of_box_end
- daemon:
- crates/minimald/src/main.rs: in pid-1 mode the daemon binds the handed proxy port and selects none
- crates/minimald/src/guest.rs: reads the handed port off the kernel command line
test: cargo nextest run -p minimald vm_hosted_daemon_binds_handed_port
produces:
- crates/sessions/src/core/switch_request.rs
consumes:
- crates/minvmd/src/box_registry.rs
- crates/minvmd/src/net/egress_gate.rs
proof: On a VM-backed host the in-VM daemon's request to publish a port or a name at the switch is applied only when the host-side table holds its address and admits the port or name, every other request is refused and logged, and the node's hostname-proxy port is assigned by the VM host daemon at boot and bound by the guest daemon as handed.
depends on:
- blocked by #1640
Proof
On a VM-backed host the in-VM daemon's publish verbs (
expose,unexpose,dns-add) reach the switch only through the VM host daemon's gate. The gate summarises each request (verb, switch address, up to four port or name records), builds the pure decision's table from its own rows, and appliessessions::core::switch_request::applied: a publish is applied when the row at the request's address holds every record, a retract when the row at its own address holds them, and everything else is refused with one rate-limited warn line naming the rule, the address, the record and the decision's own reason. The node's hostname-proxy and zone-answerer ports are resolved once inrun.rs, handed to the guest on the kernel command line, and registered in the node's row as its only admitted ports; the pid-1 daemon binds exactly those ports and selects none. A box's row is withdrawn when the shuttle relay that carried its frames ends.Interim, stated in the code.
UNREGISTERED_SOURCE_PHASEis a compile-time constant read for both the frame drop and the publish decision, so it is boot-scoped and cannot drift mid-run. While it isAnnounced, a publish or retract at a lease-run address no row holds is applied and logged on the refusal line markedinterim=true; the node address, the gateway and the reserved pair sit outside the run and are never admitted that way. Own-ip rows arrive with T66 (#1789), which also carries creator-driven withdrawal at destroy; until then a never-sending box's row is not withdrawn by this path and the address-reuse hazard (deployment-and-egress-gateway.md v0.9 §4.2) stays open. The flip toInForcebelongs to the task that lands the last row source.A box with no
egresssection. The gate installs no filter of its own for a row that carries no policy, and that is by construction, not by omission: the daemon decides the effective egress once, ineffective_egress_section(crates/minimald/src/session.rs), before it registers the box, and a box with no section arrives as an explicitdeny_allpolicy once the default is in force, as the declared section when one exists, and as no policy only when the effective verdict is allow-all (the announced phase, or the opt-out). The gate never sees an undecided box, sobox_cannot_reach_switch_apiregisters one with no policy and one with an allow-all section and both are refused at the switch's own address on TCP 443 and for ICMP while their resolver frames pass over UDP 53 and TCP 53;gate_verdict_decides_by_source_and_rulespins that a row admitting ingress port 8080 is still refused at gateway:8080.The switch's own address is a control surface. A frame from a box to the gateway address is refused on every port but the resolver's (53, TCP and UDP; DNS falls back to TCP on truncation), before any row or phase is consulted; a box's admitted ports are its own ingress and never a gateway flow, so gvproxy's API port on the gateway is unreachable whatever a box declares (
SwitchControlSurface, ruleegress-switch-control-surface, one rate-limited warn line per source naming the port). This head carries the host-side leg; the in-VM leg's infrastructure deny set already refuses the host alias and the resolver's siblings in the daemon's relay.Fail closed. A handed port token that is present but unparseable fails the boot with
HandedPortError; a handed port that cannot bind in pid-1 mode is boot-fatal. Only an absent token means the pre-handoff default-then-select path.Design review: ARCH-EXPERT PASS-with-conditions against deployment-and-egress-gateway.md v0.9 §4.3, §5.5, §7.1 and NET-138; the conditions landed in round 5 (address-keyed retract with a per-address Kani harness, interim logging, fail-closed tokens, single port resolution with
handed == registeredasserted). Bounds the gate carries that the design does not (per-table name interning, the overflow refusal class, the four-record cap) are tracked in gominimal/inbox#1004.A guest never withdraws a declared port. A
Retractis applied only for a record in the row's runtime-published set (NET-016/NET-017's listen-published ports; empty until listen-publishing lands), never for a declared port, whose forward is held for the session's lifetime and unbound only by host-side revocation (design §7.1, NET-121).SwitchRowcarries that set, the Kani contract's "admits" is per verb, andretract_of_declared_port_refusedproves it in the pure decision and through the gate.The control body is read under the same bound as the head. A guest that declares a Content-Length and withholds the body is cut off at
drain_timeoutand its relay slot released;withheld_control_body_releases_the_relay_slotdrives the real accept loop with every slot withheld and proves the next connection is admitted.The expose body's
localis shape, never decision. A summarizablelocalis127.0.0.1:<port>or<lease>:<port>on an address of the reserved local range: the two spellings the daemon's own client builds (NET-010, NET-123, NET-129), and the two published addresses the design names. Nothing on the LAN is admitted. Interim: until T66 (#1711) hands the host each row's loopback address, the gate can check shape only; once it holds the address, it verifieslocalagainst the row and refuses a request naming a sibling's reserved address, so no request can bind a forwarder on another box's address.A name-resolving row's destination rule is deferred to the guest, as an interim tracked in #1808. When a row declared
allow_dns_hosts, anUndeclaredSubnetdrop on the host is deferred to the in-VM gate, which holds the pins the box's own answers produced; the host cannot pin without resolving a second time, and a second resolution is not the box's answer (design §5.3). The arm is exactly that narrow:a_row_that_resolves_names_still_takes_every_other_drop_on_the_hostproves that for such a row the host still drops a destination in its owndeny_subnets, a frame to the switch's control surface, a protocol its rules do not allow and an unknown source. One info line per such row at registration says the rule is deferred and why. The infrastructure deny set is a host frame rule on this head, applied ahead of any row rule and ahead of the deferral, so it holds for a resolving row too: link-local and loopback always, the plane except the node's own block, and RFC 1918 unless the row'sallow_subnetscovers the destination (egress-infrastructure-destination;infrastructure_destinations_drop_on_the_host_for_every_row). It also covers a source admitted by the interim before its row registers. NET-081's verify is scoped to CIDR-only rows in the spec until #1808 moves the resolving row's destination decision host-side.Local rounds after the coder's cap. Round 10 ended on a lost runtime container with nothing pushed. The four commits after 23e7315 were made by hand: bc38df7 (the resolver-only carve-out above, from the architecture review of round 9), 838e1a0 (
render_recordcuts a target name on a char boundary;render_record_cuts_names_on_a_char_boundaryreproduces the panic), 93d2948 (the declared-port rule above, from the architecture conformance review of the spec amendments) and dcaf441 (the body-read bound above, from the review finding). Round 1 of the second cap pushed 76a1b65 (the two-spellinglocal, from the review finding) and 2bd77cc (the deferral proof and the registration line, from the architecture re-check). Local proof on macOS at dcaf441:just test-one egress_gate -p minvmd42 passed,just test(minvmd and sessions) 780 passed,just clippyclean,scripts/kani.sh10 + 3 harnesses verified; the minimald lines run in the lanes.The publish ledger keys a listener by its loopback address and port. A forward's
localnames either127.0.0.1:<port>or a reserved-range lease and port. Two boxes publishing one port at their own leases are two listeners, and each retraction is attributed to the address its own publish was applied at (two_boxes_publishing_one_port_keep_their_own_attribution).Out of layer
A tests-only hunk in
crates/minimald/src/server.rs(mod tests) drives the handed-port bind through the real listener.crates/minvmd/src/box_registry.rsgains the one info line per name-resolving row at registration (BoxRegistry::registeris the single chokepoint both register paths funnel through);scripts/kani.shcarries the harness count. No other production code outside the plan's layers changed.Tests
cargo nextest run -p sessions switch_request_applied_only_when_table_admitscargo nextest run -p minvmd switch_request_refused_and_loggedcargo nextest run -p minvmd node_port_assigned_on_host_and_handed_to_daemoncargo nextest run -p minvmd host_table_row_withdrawn_within_60s_of_relay_endcargo nextest run -p minimald vm_hosted_daemon_binds_handed_porthost_table_row_withdrawn_within_60s_of_box_endwas renamed to..._of_relay_end: it proves withdrawal on relay end, which is what this PR delivers; box-end withdrawal driven by the creator is T66's. The Kani harnesskani_switch_request_applied_iff_table_admitsruns in the Kani lane (scripts/kani.shexpects 9 sessions harnesses); the lane is green on this head.Verify
Every lane is green at the head. Base merged in before the tests ran.
Closes #1708