Repository navigation
T108 — Escapes and builds stay inside the box's rules, end to end - #2820
norrietaylor merged 9 commits into
Conversation
Add the VM-host case escapes_and_builds_stay_inside_box_rules. One deny-all own-address box makes a vsock connect toward the guest daemon's RPC port, builds a project package whose web source is outside its rules, and a spoofer wears the node's address at the host-side gate. Each leg prints its attempt, the refusal and the host-side record, and a failed leg names the record it expected and the logs it searched.
|
Important Review skippedBot user detected. To trigger a single review, invoke the
 |
|
Approved: every review thread resolved at @norrietaylor resolved the last thread, which releases the approval this review had withheld. Nothing to do. |
There was a problem hiding this comment.
arch-review: PASS_WITH_CONDITIONS · 1 condition · the full ruling
The case conforms to the design. Each leg asserts a refusal that the design states: §7.1 line 252 for the vsock door, and §4.3 rule 2 with §8 item 1 for the node-plane address. The build leg tests the spec's own-address interim, which #2802 owns. That interim is a filed residual. The proof did not run on this box, so each leg is open. C1 requires the required KVM lane to pass all three legs at this head before merge.
Ruled at 8dd7f17 against gominimal/arch@8640f85e193d by claude-opus-5-5.
The gate writes one drop line per source address per rule in a 60 s window, whatever the destination. An earlier drop from the node's address can silence the build leg's expected line, so wait 61 s first, as the escape leg does.
A Linux host without KVM boots no VM, so the case has no host-side gate to read. Skip there as the sibling VM-backed cases do. A KVM host that lacks MINVMD_GVPROXY_BIN still fails the case.
|
Round 1 of 10 on the runtime's review findings, pushed as
Verify Review threads the coder judged:
On a VM host, the case escapes_and_builds_stay_inside_box_rules shows that a box never reaches the guest daemon's RPC listener over vsock. It shows that a deny-all box's package build and root in the VM wearing the node's address reach nothing outside the box's rules and the baseline set. It prints each refusal beside its record in a host log. This round, the build leg first waits 61 s for the gate's per-source, per-rule line window to close, so an earlier drop from the node's address cannot silence the line the leg expects. This was not observed on this box, because the box has no VM. Notes from the agent: Revise round 1. The one finding holds: DropLimiter keys on source address and rule (test drop_limiter_rate_limits_per_source_and_rule in crates/minvmd/src/net/egress_gate.rs), not on destination. I added an unconditional 'sleep 61' before the build leg, with a printed line that says why it waits. A conditional wait needs the gate lines' timestamps, and the script does not pin their format. A drop from an earlier case before the log snapshot can also hold the window open, so a check of only the case's log lines is not enough. The cost is 61 s more e2e time. The test line is not proved here: this box has no KVM and no MINVMD_GVPROXY_BIN, so the case exits at its precondition. The KVM lane (just e2e with E2E_VM=1) still owes the evidence. bash -n and shellcheck -S warning are clean. |
There was a problem hiding this comment.
arch-review: PASS_WITH_CONDITIONS · 2 conditions · the full ruling
The case still conforms to §7.1, §4.3 rule 2 and §8 item 1, so the baseline ruling stands. The new /dev/kvm gate prints SKIPPED and returns 0. The run then ends session e2e OK, and the case is not in the NOT RUN count. On a host without KVM, the task's test line now passes and runs no leg. C1 changes that gate to not_run. No leg has run on any head, so C2 asks for one VM-lane run at this head, with its leg table in the PR.
Ruled at d94621c against gominimal/arch@8640f85e193d by claude-opus-5-5.
The in-box vsock refusal now needs socat's own socket(AF_VSOCK) error and a non-zero exit, and its row says it has no host-side record. A socket that opens must produce a new guest refusal line for that attempt. A build that hangs to timeout 124 is a fault. The build leg names issue #2802 as the owner of the node-address interim.
|
Round 2 of 10 on the runtime's review findings, pushed as
Verify Review threads the coder judged:
On a VM host with KVM, the case escapes_and_builds_stay_inside_box_rules shows each leg's attempt beside its refusal. A vsock attempt from the box never reads the RPC listener's SSH banner. A deny-all box's package build and a frame from root wearing the node's address each get an egress-node-plane-unpinned line in the host log. This round did not show that on this box, because the case skipped here without /dev/kvm. Notes from the agent: This box has no /dev/kvm and no MINVMD_GVPROXY_BIN, so the case skipped and no leg ran. The ci-linux-kvm lane owes the evidence. shellcheck is not installed here, so it did not run. The task proof says each refusal is recorded in a log on the host. The vsock refusal inside the box (EAFNOSUPPORT) leaves no host record, and the case now says so in its row and its final line. Only the guest refusal path, when a vsock socket opens, writes a host-side boot-log line. |
There was a problem hiding this comment.
minimal-review — approval withheld
Verdict COMMENT — APPROVE withheld: 8 unresolved threads — minimal-foundry[bot] scripts/session-e2e.sh:20505, minimal-arch[bot] scripts/session-e2e.sh:20146, minimal-arch[bot] scripts/session-e2e.sh:20124, minimal-foundry[bot] scripts/session-e2e.sh, minimal-foundry[bot] scripts/session-e2e.sh, …. No findings.
Panel: 3 of 3 personas ran (correctness, security, supply-chain).
There was a problem hiding this comment.
arch-review: PASS_WITH_CONDITIONS · 2 conditions · the full ruling
The case conforms to §7.1, §4.3 rule 2, §5.1 and §8 item 1, and each leg fails closed. The compare tightens the vsock and build checks. It also names #2802 as the owner of the build interim. Two conditions stay open. C1: run the case on a KVM or macOS lane with MINVMD_GVPROXY_BIN set, and add its table to the PR. C2: amend the T108 proof line in #2704. The box's own socket(AF_VSOCK) refusal leaves no record on the host, as the case already says.
Ruled at 9f97785 against gominimal/arch@8640f85e193d by claude-opus-5-5.
The no-KVM gate printed SKIPPED and returned 0, so the lane summary ended OK with no NOT RUN line. It now calls not_run, so the summary names the case as not run.
There was a problem hiding this comment.
arch-review: PASS_WITH_CONDITIONS · 1 condition · the full ruling
The compare conforms. On Linux without a writable /dev/kvm, the case now counts as NOT RUN in the lane summary, not as an uncounted skip. The legs and their records do not change, so the ruling on 9f97785 stands for them. No host ran the legs yet. The Tests row stops at the switch-binary gate with exit 1. C1 asks for a passing row from a KVM or macOS lane with the switch. Section 7.1 line 252 and section 8 line 315 are the rules that row proves.
Ruled at c5c6331 against gominimal/arch@8640f85e193d by claude-opus-5-5.
|
Round 3 of 10 on a failed check suite, pushed as
Verify Review threads the coder judged:
Not yet shown at this head. On a VM host (KVM or macOS with MINVMD_GVPROXY_BIN), the case runs three legs: the RPC leg, the build leg and the escape leg. Each leg prints its attempt, the refusal and the host-side record it found. The in-box socket(AF_VSOCK) refusal is the one refusal with no host-side record, and its row says so. On a host without KVM, the lane summary now names the case as NOT RUN. Before this change, the case was silently skipped. Notes from the agent: This round changed one line: the no-KVM gate now calls not_run in place of the SKIPPED echo. This box has no /dev/kvm, so no leg ran here. The VM run is still owed by ci-linux-kvm or the macOS e2e lane. The failed e2e job excerpt does not name the failing case. The excerpt shows only other cases' expose lines and the runner cleanup, so I could not tell if this case failed on macOS. That lane sets MINVMD_GVPROXY_BIN, so the switch gate does not explain it. Read the macOS e2e job log for a line that starts with 'escapes_and_builds_stay_inside_box_rules' or '::error::' inside this case's group. ci-macos-success is the aggregator and fails because e2e failed. The native-minimald gate (min_daemon is not minvmd and E2E_VM is not set) still prints SKIPPED and returns 0, as the thread asked only for the KVM gate. |
There was a problem hiding this comment.
minimal-review — approval withheld
Verdict COMMENT — APPROVE withheld: 11 unresolved threads — minimal-foundry[bot] scripts/session-e2e.sh:20505, minimal-arch[bot] scripts/session-e2e.sh:20146, minimal-arch[bot] scripts/session-e2e.sh:20124, minimal-foundry[bot] scripts/session-e2e.sh, minimal-foundry[bot] scripts/session-e2e.sh, …. No findings.
Panel: 3 of 3 personas ran (correctness, security, supply-chain).
|
Revise round 3 ran against |
…d-builds-stay-inside-the-box-s-rules-e
The RPC leg took its baseline count of the guest daemon's refusal lines after the vsock attempt had run, so the attempt's own line was already counted and the wait for a new one could never succeed. Count before the attempt, and match only lines that name the attempt's peer CID.
|
Round 4 of 10 on a failed check suite, pushed as
Verify Review threads the coder judged:
Not delivered on this box. On a KVM or macOS VM lane with MINVMD_GVPROXY_BIN set, the case prints each leg's attempt beside its refusal and the host-side record. It prints 'RPC leg OK', 'build leg OK' and 'escape leg OK': no box reaches the guest daemon's RPC listener over vsock, and the deny-all box's build and a frame from the node's address reach nothing outside the box's rules and the baseline set, each with a gate line in the VM host daemon's log. This box has no /dev/kvm, so the case reports NOT RUN and nobody has seen these lines yet. Notes from the agent: Round 4. The failed-check excerpts for e2e, test-kvm and the two lane aggregators do not show this case's output, so the directive does not say which line failed. While I read the case, I found a real defect in the RPC leg. The baseline count of the guest daemon's 'refused an RPC connection from a vsock peer' lines was taken after the vsock attempt ran. If the socket opened, the attempt's own refusal line was already in the count, and the 10 s wait for a new line always failed. The count is now taken before the attempt, and only lines that name peer_cid=<the attempt's CID> count. This defect is a possible cause of the KVM failure, but I cannot confirm it from the excerpts. I did not run the case on a VM: this box has no /dev/kvm. A person must read the e2e and test-kvm job logs at the new head for 'RPC leg OK', 'build leg OK' and 'escape leg OK'. If a leg fails again, its ebr_fail output names the record it expected and the host log lines it searched. shellcheck is not installed here. I checked the syntax with 'bash -n' and checked the new peer_cid regex against sample console and JSON lines. |
There was a problem hiding this comment.
arch-review: PASS_WITH_CONDITIONS · 1 condition · the full ruling
The compare changes only the RPC leg. It counts the guest daemon's refusal lines before the vsock attempt and matches the dialled CID. The leg stays hard and fails closed, as §7.1 line 252 and NET-139 need. The case has still not run on a VM lane. The only run stops at the switch-binary gate. C1 asks for one green run on a KVM or macOS lane, with its table in the PR.
Ruled at ed1ee0b against gominimal/arch@8640f85e193d by claude-opus-5-5.
There was a problem hiding this comment.
minimal-review — approval withheld
Verdict COMMENT — APPROVE withheld: 13 unresolved threads — minimal-foundry[bot] scripts/session-e2e.sh:20513, minimal-arch[bot] scripts/session-e2e.sh:20146, minimal-arch[bot] scripts/session-e2e.sh:20124, minimal-foundry[bot] scripts/session-e2e.sh, minimal-foundry[bot] scripts/session-e2e.sh, …. No findings.
Panel: 3 of 3 personas ran (correctness, security, supply-chain).
|
Revise round 4 ran against |
|
/revise Both e2e lanes fail the same leg of the T108 case on ed1ee0b: macOS Evidence (macOS lane):
The listener sits on the runner's own address ( Fix, in
Do not edit the PR body. |
…d-builds-stay-inside-the-box-s-rules-e
The listener sits on the runner's own address, an infrastructure destination, and the gate checks that deny set before the node-plane pin. Accept either rule there, and add a second probe from the node's address to the TEST-NET-2 address to assert egress-node-plane-unpinned itself.
|
Round 5 of 10 on a /revise note, pushed as
Verify Review threads the coder judged:
On a VM host, the case escapes_and_builds_stay_inside_box_rules shows the following. No box reaches the guest daemon's RPC listener over vsock. A deny-all box's package build fails, with an egress-node-plane-unpinned gate line for its web source. A frame from root in the VM that wears the node's address is refused toward the runner's own address. The gate records it under egress-infrastructure-destination or egress-node-plane-unpinned, and the listener gets no marker. A second such frame toward 198.51.100.9:443 is refused with an egress-node-plane-unpinned line. This box has no KVM, so this round did not observe the proof. The KVM and macOS e2e lanes must show it. Notes from the agent: Revise round 5: the spoofed-node-address leg now does what the note asks. (1) After a 61 s wait, the probe toward the listener on the runner's own address must be refused with no marker at the listener. It also needs a new gate line with source = node address and destination = that address, under egress-infrastructure-destination or egress-node-plane-unpinned. The leg counts the prior lines for each rule separately. (2) After a second 61 s wait, a new probe from the node's address toward 198.51.100.9:443 must be refused, and a new egress-node-plane-unpinned line must name that source and destination. I updated the case header comment and the final summary line to match. The test line ran on this box and exited 0, but the case was NOT RUN because there is no writable /dev/kvm. The legs still owe evidence from the ci-linux-kvm and macOS e2e lanes. |
There was a problem hiding this comment.
arch-review: PASS_WITH_CONDITIONS · 1 condition · the full ruling
The new commit is correct. The gate checks the infrastructure deny set before the node-plane pin, so probe 1 can drop under either rule. Either rule keeps the node's address inside the baseline set, which §4.3 line 84 and §8 line 315 require. Probe 2 aims at an address that only the pin can drop, so the pin is still proved by name. One item stays open from the earlier heads: the case has never run on a VM lane. C1 closes it with a passing run on a KVM or macOS lane and a row in the PR body.
Ruled at eb37350 against gominimal/arch@8640f85e193d by claude-opus-5-5.
There was a problem hiding this comment.
minimal-review — approval withheld
Verdict COMMENT — APPROVE withheld: 14 unresolved threads — minimal-foundry[bot] scripts/session-e2e.sh:20519, minimal-arch[bot] scripts/session-e2e.sh:20152, minimal-arch[bot] scripts/session-e2e.sh:20124, minimal-foundry[bot] scripts/session-e2e.sh, minimal-foundry[bot] scripts/session-e2e.sh, …. No findings.
Panel: 3 of 3 personas ran (correctness, security, supply-chain).
There was a problem hiding this comment.
Babysitter review of eb37350 against T108 (#2704). The case passes on both VM lanes:
RPC leg OKbuild leg OK, after a 61 s gate window, and failing on exit 124escape leg OK: an infrastructure-destination probe and a TEST-NET probe that hitsegress-node-plane-unpinned
The PR body records the leg table. I amended the proof line in #2704 so it does not claim a host-side record for a vsock socket the box refuses.
There was a problem hiding this comment.
minimal-review — approved
Verdict APPROVE — every review thread is resolved; @norrietaylor closed the last one. The review of eb37350a141f withheld this approval on the threads open then. Posted as a comment: the App cannot approve its own pull request; the check and the label carry it.
T108 — Escapes and builds stay inside the box's rules, end to end
slice: S16
kind: integration
covers: none
size: M
est: 60 code, 200 tests
executor: agent
difficulty: routine
model: standard
diagnostics: a failed leg prints which refusal or host-side record it expected and the host log lines it searched
observability: the case prints each leg's attempt beside the refusal and the host-side record it found
layers:
- e2e:
- scripts/session-e2e.sh: adds the case escapes_and_builds_stay_inside_box_rules, covering root inside the VM sending from the node's own address, a deny-all box's package build and the host-side records
test: ./scripts/session-e2e.sh escapes_and_builds_stay_inside_box_rules
produces: none
consumes:
- crates/op/src/specs.rs
proof: On a VM host, a process inside the VM never reaches its RPC listener from another box, root inside the VM and a package build run for a box reach nothing outside the box's rules and the node-plane baseline set, and each refusal is recorded in a log on the host.
depends on:
- blocked by #2690
- blocked by #2691
- blocked by #2692
- blocked by #2693
- blocked by #2694
- blocked by #2730
Generated by Claude Code
Proof
On a KVM or macOS lane with the switch binary, ./scripts/session-e2e.sh escapes_and_builds_stay_inside_box_rules starts one deny-all own-address box and runs three legs. (1) A vsock connect from inside the box to the guest daemon's RPC port fails. It fails at the socket(AF_VSOCK) seal, or else at the daemon's accept check, whose audited line the case reads from the host-side boot log. The box never reads the SSH banner. (2) A build of a project package whose web source is outside the box's rules fails, and the case reads the host-side gate's egress-node-plane-unpinned line for that destination from the minvmd log. (3) A spoofed frame that uses the node's own address toward the runner's address is dropped. The listener gets no marker, and the case reads the gate's egress-node-plane-unpinned line that names the source and the destination. The case prints a table that shows each attempt, its refusal and the host-side record. I could not show this on this box, because it cannot boot a VM.
Notes from the agent: Not verified here. This box has no /dev/kvm, no libkrun and no MINVMD_GVPROXY_BIN, so the test line stops at the case's switch-binary gate, the same lane-fault gate the escape-union case uses. The case needs a KVM or macOS lane. It is in the VM-default list at the top of the script, so the test line sets E2E_VM=1 and --provider local-minvmd by itself. It also runs in the whole-lane order, directly after escape_reaches_only_declared_union. Each design point below is an assumption that only a VM run can confirm: (a) The RPC leg expects EAFNOSUPPORT, because the sandbox seals admit no AF_VSOCK. That refusal is recorded only as the box-side error. The guest daemon's CID audit line (NET-139) can be on the host, in the boot log, only when a socket opens, so the case requires that line only in that branch. The case uses the box's socat (VSOCK-CONNECT) and port 2222 (instance 0). (b) The build leg runs in-box 'min add --session e2e-ebr-fetch' on a project-loc…[truncated]
Tests
./scripts/session-e2e.sh escapes_and_builds_stay_inside_box_rules,test-kvmlane at eb37350 (run 41257763642, job 124705927242)escapes and builds stay inside the box's rules OK. NET-085 NET-139 NET-155e2elane at eb37350 (run 41257763653, job 124706334605)escapes and builds stay inside the box's rules OK./scripts/session-e2e.sh escapes_and_builds_stay_inside_box_rulesThe attempt, refusal and record table from the
test-kvmrun (gate records shortened to their rule):socket(AF_VSOCK)refused inside the box (EAFNOSUPPORT)socket(AF_VSOCK)refused inside the box (EAFNOSUPPORT)e2e-ebr-fetch, web source198.51.100.9:443egress-node-plane-unpinned, destination198.51.100.9100.64.255.253(the node's address) toward10.1.0.138:18095egress-infrastructure-destination, source100.64.255.253, destination10.1.0.138100.64.255.253toward198.51.100.9:443egress-node-plane-unpinned, source100.64.255.253, destination198.51.100.9The earlier row with exit 1 is the coder's sandbox, which has no
MINVMD_GVPROXY_BIN.Verify
min task run agent-verifypassed ([tasks.verify]).Closes #2704