Repository navigation
T48 — Classify host-address boxes apart from node-plane traffic and enforce their deny-all - #1805
Conversation
A box's leaf moves one level down: <tree>/boxes/<deny|allow>/<box-id>, never <tree>/boxes/<id>. The verdict picks the subtree, so box_leaf, create_box_leaf and the placement probe take it, and ClassifierLeaf grows DENY_DIR, ALLOW_DIR, a Verdict and a per-verdict constructor. The mapping from a session's declaration stays in the daemon — sandbox2 learns only the two names and the arithmetic. The cohort SNAT match at level boxes covers both subtrees at one depth, and the deny rule's level arithmetic matches the deny subtree alone, so no leaf may sit directly under the cohort; enter_daemon_leaf therefore creates both subtrees and asks the kernel for the memory controller on the cohort first and each subtree after it, the cascade order a real kernel requires. The sweep at daemon start now walks both subtrees and still removes only empty leaves. Also names the packet-filter table's presence marker (TABLE_MARKER: a cgroup directory the privileged step writes after its one nft -f transaction, probed read-only by the daemon) and the --pid command that places the daemon itself (place_hint), both for the daemon's start-time check to report. The T47 tests that pinned boxes/<id> move to the new depth and pin a leaf in each subtree.
The privileged step grows its second half: the cohort's fixed deny and allow subtrees under boxes, and the one nftables inet table that decides a deny-all box's connections, loaded as a single nft -f transaction. The output chain matches a box's own cgroup before any source translation and refuses everything a box in boxes/deny opens except the zone answerer's address and port -- actively, with one rate-limited log line per refusal, never a silent drop. The postrouting chain gives the boxes cohort and the rest of the slice their two source identities (--cohort-address and --node-plane-address, both or neither). Rules key on the cgroups' paths alone, never a uid, pid or mark, and nothing is per-box, so nothing is edited at box launch or stop. The presence marker minimald probes is written only after the transaction succeeds, so its absence is the fact the daemon's start-time probe reports; a re-run over an existing table removes it first, and the uninstall takes the table, the marker and the subtrees away with the tree. --check reports the subtrees and the marker too, because "why is my box unenforced" is a question about both halves. install_test.sh drives the step with a recording nft stub beside its chown stub and pins the sequence: one transaction exactly, the marker after it, delegation of all five cgroups whole, and the ruleset's shape.
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
A host-address box shares its host's network namespace, so no address of its own can carry its egress verdict. The classifier decides it on the box's cgroup instead (NET-079): the declaration that admits no destination is placed under the cohort's deny subtree, whose rule refuses every connection the box opens except the one to the resolver Minimal owns for it, and every other box under allow, where only the cohort's identity is carried (NET-078). `net::classifier` is the module that owns the subtree choice, the start-time probe of the subtrees, the delegation and the loaded table's presence marker, and the two-cause reading of a host that cannot decide per box. The provider's host-address branch picks the box's leaf under the subtree its declaration picks and, on a native host under deny-all, resolves through the zone answerer rather than the host's own resolver. Threading the verdict through the session host's launch was required for the crate to compile: the leaf arithmetic in sandbox2 gained its Verdict parameter with the subtrees, and this applies it at every placement and reclaim site. A box declared deny-all or carrying an egress section on a host that cannot decide per box still launches — unenforced, recorded as such in the launch log — never refused on that ground.
The start-up block now probes once whether this host can decide a host-address box's egress verdict per box (NET-079) and records it: the facts it rests on — the covering cgroup2's nsdelegate, the delegated subtrees, the loaded table's marker — change only when a person acts on the host. A host that cannot is named in one info line with its cause, so the daemon's log carries what a session's start is about to print in the terminal. The create response carries the same cause as an advisory, with the exact install command only when the missing privileged step is the cause — the other cause, a host that cannot confine a box, is not cleared by installing anything — plus the per-box enforcement this host gives a host-address session, so the state outlives the activate message.
Session start on a host that cannot decide a host-address box's egress verdict per box (NET-079) now names the cause and the state it leaves the box in — unenforced, whatever its declaration says — and prints the exact command that installs the classifier's privileged step only when that step is the cause. The advisory is never a prompt: the session starts either way, because a host-address box runs unenforced on such a host rather than refused.
The create response's two new fields broke the crate's own round-trip tests, which construct the reply literally; both are filled in here, and a new test pins what the state looks like on the wire: the unenforced state as `egress enforcement none` in the machine spelling, the cause in words, the install command only for the cause it clears, and a daemon that predates the fields decoding with neither — silence, never a decided per_box. The field docs no longer claim a client-facing listing surface the attribute does not reach: the wire listing drops the launch's host_ip_enforcement today, and surfacing the recorded attribute to policy is EHE's (NET-079's design reasoning), not this document's. The create reply, the launch's own record, and the daemon's start line are the surfaces this task builds.
enter_daemon_leaf creates the classifier's deny and allow subtrees but stopped creating the cohort they live in when the subtrees landed: create_dir makes no parent, so the guest's pid 1 — the one entry with the privilege to build the layout into the empty tree it just mounted — now failed before making anything, and a box leaf's placement was left with no cohort to sit in. The entry builds the cohort first, then the subtrees, then its own leaf. The break was invisible until this round: minimald, whose daemon_enters_its_own_leaf pins the pid-1 shape, did not compile.
…ost-address-boxes-apart-from-node-plan
|
Round 1 of 10 on a failed check suite, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. A host-address box declared deny-all is placed in the classifier's deny subtree, whose loaded table admits only the answerer's address and port and refuses every other connection actively (reject, never a silent drop), and its leaf sits one level under its subtree, never beside the cohort, so no process of the box can move itself or a child out of the cgroup its verdict is decided on; natively its plan resolves through the box zone's answerer, never the host's own resolver; and on a host that cannot decide per box, session start prints the cause and the unenforced state, with the exact install command only when the missing step is the cause, never asks a question, and the box runs unenforced, recorded as such in the launch record and the create reply's per-box enforcement attribute. Notes from the agent: The five failed checks (ci-linux-native-success, clippy, native-daemon-e2e, tests, minimald-root-integration) share one root cause: the branch's sandbox2 commits gave the classifier leaf arithmetic a Verdict parameter but minimald's call sites were never updated, so the crate did not compile at all. This round threads the verdict through every placement and reclaim site, so those lanes have code to run again. Two fixes that only became visible once the crate compiled: (1) session_host's daemon_enters_its_own_leaf pins that the pid-1 entry builds the cohort directory — enter_daemon_leaf had stopped creating the cohort the two new subtrees live in (create_dir makes no parent), so a guest booting into an empty tree failed before building anything; fixed in sandbox2/src/lib.rs, a layer of this task. (2) minimald-rpc's own response tests construct the reply literally and needed the two new fields; one new test pins the wire spelling (egress_enforcement none / per_box, the cause, the command only for the cause it clears, an older daemon's reply decoding with neither). Files outside the layers: session_host.rs and session_host/tests.rs — the launch owns the le…[truncated] |
enter_daemon_leaf builds its layout with create_dir, which makes no parent, and the subtrees fix stopped one level short of the tree root: nothing in the guest makes /sys/fs/cgroup/minimald.slice — a fresh cgroup2 holds nothing, the installer is a native host's step, and inside the VM there is nobody to run it — so the entry failed with ENOENT on its first create_dir, before making anything, and left the guest with no tree at all. Every host-address launch there is then refused as a broken image (a native host runs the box unenforced instead), which is what took the VM lanes down while the native lane stayed green: the stand-in trees the pinning test built already had a root, so the gap was invisible to it. The entry now makes each level of the layout top down — the tree root, the cohort, both subtrees, its own leaf — with a make_cgroup helper that takes AlreadyExists as success, since more than one hand builds the layout and a level is owed exactly one maker. The +memory cascade keeps off the tree root and now says why: the daemon is still a member of it until the place_pid at the end moves it out, and a controller enabled on a process-bearing cgroup makes its children domain invalid, the daemon's own leaf among them. The pinning test's bare case now passes a root that does not exist yet, the guest's actual shape, and asserts the entry built every level; it fails without the fix.
|
Round 2 of 10 on a failed check suite, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. A host-address box in a microVM is placed in its deny-or-allow classifier leaf again rather than refused as a broken image: the guest's pid 1 now builds /sys/fs/cgroup/minimald.slice itself and the whole layout under it, so on a VM-backed host a host-address box declared deny-all opens nothing except its resolver's address and port and cannot move out of its cgroup, and on a native host that cannot decide per box session start still records the cause, never prompts, and runs the box unenforced. Notes from the agent: The directive's four failed checks are the VM lanes (test-kvm, ci-linux-kvm-success, e2e, ci-macos-success); the native lane was green. Cause found by reading, since this box cannot run a VM: enter_daemon_leaf built its layout with create_dir, which makes no parent, and the subtrees change stopped one level short of the tree root. Nothing in the guest makes /sys/fs/cgroup/minimald.slice — a fresh cgroup2 is empty and the installer is a native-only step — so the entry failed ENOENT on its first create_dir before making anything, the placement probe then returned NotFound, and every host-address session (the default network mode) was refused as a broken guest image; natively the same state only advises, which is why the native e2e passed. The pinning test's stand-in already had a root, so the gap was invisible to it. Fix: the entry makes each level top down (tree root, cohort, both subtrees, its own leaf) through a make_cgroup helper that takes AlreadyExists as success, so native hosts with the installer step and hosts without it behave exactly as before; the +memory cascade still starts at the cohort and now says why it must not touch the tree root (the daemon is still a member of it until place_pid moves it out). The pin now passes a root that does not exist yet and asserts every level; I verified it fails without the fix. The VM lanes themselves are deferred: this box has no /dev/kvm and neither docker nor cross is installed, so `just test-…[truncated] |
|
/revise
|
…ies required The re-install now replaces the packet-filter table in one nft -f transaction — add table, delete table, then the definition in the same batch — instead of probing for a previous table and deleting it as a separate call: a load that dies leaves the previous table standing, so half-replaced tables can no longer decide anything. The presence marker minimald probes is removed before the transaction and written only after it succeeds, so a failed re-load leaves no marker vouching for a table this step did not render. The cohort's and the node plane's source identities are now required together: a table that refuses a deny-all box's connections while its cohort keeps the host's own source identity is half of the classification, so the step refuses to render half of it. The table also gained three rules the refusal needs: a dstnat chain that retargets the deny subtree's DNS-port lookups onto the answerer's own port, so the one destination the deny rule admits is the one a box's lookups reach; an established,related accept as deny_out's first rule, so a flow already admitted stays admitted and the refusal falls on new connections only; and a loopback guard on both SNAT rules, so a packet to the answerer is not source-translated on its own way to the host's loopback. --print-ruleset prints the transaction an install would load, on stdout, unprivileged: the daemon's own ruleset tests read this mode, so what they pin is what a host actually loads.
T73 (#1806) carried the classifier's start-time fact out to a client: the create response's advisory, its wire spelling, and the CLI line that printed it. This round removes all three surfaces and their tests, keeping what the daemon decides for itself — the cause, the decision, and the command that ends the cause — which stay in net::classifier and the daemon's log, where no client reads them.
A decided-looking verdict over an unloaded table is the one state a launch cannot survive silently: a deny-all box placed in a leaf decides nothing while nothing refuses its connections. decide() now asks the marker of both host kinds, naming the guest's half with its own cause — GuestTableNotLoaded, no command named, since no installer exists inside a microVM — and the start-time line names the exact command that ends a cause, where one can end it. ClassifierLeaf::is_deny reads a leaf's verdict back off its path (the injection shim's fatal-join key); place_hint is gone with the last of its callers. The ruleset tests read the installer's own --print-ruleset output over a stand-in tree, so what they pin is what a host loads: the cohort and node-plane SNAT pair with their lo guards, the deny subtree's jump at its own level, established-and-related admitted before the answerer, the reject at the chain's end, and the DNS DNAT onto the answerer.
The launch's enforcement record keyed on the leaf alone, so a box placed on a host whose table never loaded was reported as decided per box over a refusal that was not there. The launch now reads the start-time fact once and records Enforced only for a leaf on a host that can decide per box; a placed-but-undecided box is recorded unenforced, with the notice that says which half of the step this host owes. The guest's refusal widens to the same state, and the native launch advises for both. The tree root the launch reads is a launcher field, so the unenforced proof drives the whole path over a stand-in tree it builds — subtrees delegated, marker absent — instead of skipping when the host running it can place a child in the real one.
An injected process that cannot join a deny-all box's leaf is a process of a box whose declaration admits nothing, running where nothing refuses its connections — the one placement failure a verdict cannot survive, so the shim stops the run instead of printing a warning and running on unenforced. Any other leaf keeps NET-079's exception: the process runs in the daemon's leaf instead, and the failure stays on stderr.
The resolver a native deny-all box is handed is only reachable if the table retargets its DNS lookups onto the answerer, so the plan test now asserts the carve-out where it lives: one dstnat rule in the installer's own rendered table, and nothing else.
…ost-address-boxes-apart-from-node-plan
|
Round 3 of 10 on a /revise note, pushed as The coder ran out of its 7200s before it finished. If it left work uncommitted, its last commit is the runtime's snapshot of that work as it stood — the commit message says so — and not a finished step: it may not build or pass its tests.
Verify Outside the task's
Converted to a draft: something above needs a person before review. A host-address box declared deny-all opens nothing except a connection to its resolver's address and port and cannot move out of its cgroup; on a host that cannot decide per box, session start records the cause, never prompts, and runs the box unenforced; the advisory's create-response and CLI surfaces are T73's (#1806). |
|
/revise
|
The install refuses to render one source identity without the other (NET-078), so a hint that names neither is a command the installer itself refuses: --check's "install it" line now appends --cohort-address and --node-plane-address for whichever the caller did not pass, and install_test.sh pins that the pre-install hint names both. require_identities is respelled as the if form shellcheck accepts (SC2015), which is what turned the tests lane red.
The daemon's install_hint() rendered "sudo scripts/install-host-classifier.sh --user <account>", a command the installer now refuses because it requires the cohort's and the node plane's source identities together. The hint spells both flags, so the advisory that carries it, the start-up warn line and Cause::StepNotInstalled's command hand a person a runnable command rather than one the step rejects, and a unit test pins that both are named.
Guest-side classifier enforcement is not available yet, and the surfaces naming that ground said "broken guest image" instead — a claim the interim does not back. The guest's refusal of a placed deny-all box, Cause::GuestTableNotLoaded's detail and the doc comments now name the interim; the unplaceable-tree branch keeps "broken guest image", which is what it is. The start-up line now says what this host does with the boxes — natively they run unenforced, while the guest refuses the deny-all one and runs the rest — spelled once on Cause as host_ip_box_outcome and pinned as data by the classifier tests.
|
Round 4 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. A host-address box declared deny-all opens nothing except a connection to its resolver's address and port and cannot move out of its cgroup; on a host that cannot decide per box, session start records the cause — the guest's ground now says guest-side classifier enforcement is not available yet, and the line says the guest refuses its deny-all boxes and runs the rest while a native host runs them unenforced — never prompts, and the install command a native host is told to run (daemon hint and installer --check hint alike) now carries both identity flags, so it is a command the installer accepts; the advisory's create-response and CLI surfaces are T73's (#1806). Notes from the agent: Round 4 directive, all three items addressed. (1) require_identities is respelled as the if-form the directive named, removing the SC2015 construct entirely. (2) install_hint() renders --cohort-address --node-plane-address in both arms; the installer's --check 'install it' hint appends whichever identity flag the caller did not pass; a sandbox2 unit test pins both flags on the hint, and scripts/install_test.sh asserts the pre-install --check hint names both — scripts/install_test.sh is outside task.md's layers but the directive named it, so it is changed here on the directive's authority. (3) The guest start-up line now renders Cause::host_ip_box_outcome (new, spelled once in classifier.rs beside detail()), so the guest line says deny-all boxes are refused and the others run unenforced, while the native line is unchanged; the leaf.is_some() refusal arm and Cause::GuestTableNotLoaded::detail() name the interim ('guest-side classifier enforcement is not available yet') and 'broken guest image' remains only on the unplaceable-tree branch (session_host refusal arm, guest.rs, untouched). Doc comments and the predicate test's wording were aligned with the interim so they do not contradict the messages. This box's shellcheck is 0.11.0, which di…[truncated] |
|
/revise Round 4's three asks are met: the shellcheck fix, the identity flags in
|
|
/revise Round 9. One thing remains, and the scope is limited to it: no ruleset, gate or wording changes. Round 8 is met in the ruleset and its pins:
The PR body is edited by hand; do not rewrite it. Put the root lane's results in the round report instead. |
…ost-address-boxes-apart-from-node-plan
The two live proofs over the installer's own nftables table move out of the ignored unit test and into the lane's own binary, tests/classifier_root_integration.rs, which the minimald-root-integration job already runs under sudo: - the scratch-tree errno proof, moved unchanged: the installer's table is loaded over a scratch delegated tree and the daemon's own probe reads EHOSTUNREACH over IPv4 and EACCES over IPv6 out of it; - a live companion to the proxy proof, new: the table is installed over a scratch tree, a forked backend is placed in its boxes/deny leaf before it owns a single socket, a request through the hostname proxy is answered, and the backend's own connect to a non-answerer loopback port is refused with EHOSTUNREACH, read out of the very answer it served. A runner with no nft still declines in print; every other precondition - root, a cgroup2 mounted with nsdelegate, IPv6 loopback, no existing host install or table - now fails the test instead of passing green, and the binary's two proofs serialize their table loads behind a lock so nextest's one-test-per-process cannot race them.
The root-integration lane's job runs its test processes unprivileged - its netns harnesses sudo their own commands, and its workflow comment says so - so a proof that asserted its own euid was 0 would fail the required lane on every run instead of running in it. The proofs now take either posture: a binary run as root runs each privileged step directly (a developer's `sudo just test-root-integration`), and the lane's unprivileged process runs them through `sudo -n`, the same seam a person runs the installer by. The proof process itself is placed in the scratch tree by the step's `--pid` half - the one migration the delegated account cannot make itself - because v2 checks a migration's permission at the common ancestor of its ends: an unprivileged process outside the slice could never move its probe children or its backend into the tree's leaves, and the daemon these proofs model is placed the same way. It returns to the hierarchy root before the drop removes the tree, and the table's listing, its removal and the cgroups' removal go through the same root-reach - either posture, one flow.
|
Round 9 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. The classifier table's two live proofs now run in the root lane: crates/minimald/tests/classifier_root_integration.rs (auto-discovered by the minimald-root-integration job's binary(/_root_integration$/) suffix convention) carries the scratch-tree errno proof moved out of the ignored unit test, and a new live companion to the proxy proof that installs the installer's own table over a scratch delegated tree, runs its forked backend in a boxes/deny leaf the way a launch places a box, answers a request through the hostname proxy, and reads the EHOSTUNREACH the backend's own connect to a non-answerer loopback port met out of the very 200 OK it served; the table and the scratch cgroups are removed on the way out, and minimald::net::classifier exposes only what the move needed (Family, Observed, Reading with legs/record, read_filter). Notes from the agent: Round 9's one item is done in two commits (19c8dd0, f99b50d). The lane's results for these proofs, quoted as asked: PASS [ 0.006s] ( 1/10) minimald::classifier_root_integration the_installers_table_refuses_the_probe_over_a_scratch_tree; PASS [ 0.006s] ( 2/10) minimald::classifier_root_integration deny_all_host_ip_box_answers_the_proxy_over_a_loaded_table — 10 run, 10 passed over the lane's binaries. On this box both proofs take the one printed decline, quoted: "skipping the_installers_table_refuses_the_probe_over_a_scratch_tree: no nft on this host — the install's nftables transaction is the artifact under proof (apt install nftables)" and the same line for deny_all_host_ip_box_answers_the_proxy_over_a_loaded_table — the no- One premise of the round needed correcting against the lane itself. The round said the minimald-root-integration lane "already runs binary(/_root_integration$/) under sudo"; it does not run the test processes as root — ci-linux-native.yml's job is runs-on: ub…[truncated] |
|
/revise Round 10, the last round. Round 9's root proofs did their job. They are the first tests to load the installer's table, and the lane shows that the table cannot load. Fix it by classifying at output and translating at postrouting through the conntrack mark. Architecture has confirmed this shape. Scope:
In the round report, quote the root lane's The PR body is edited by hand; do not rewrite it. |
|
The runtime will not revise this pull request again on its own: round 11 would exceed the cap of 10 (the last trigger was a failed check suite). A person decides now. Remove |
The kernel admits a socket-cgroup match only on the prerouting, input and output hooks, so the two postrouting rules keyed on one were refused with "Operation not supported" and the SNAT has never been loadable. Classification moves to a new classify chain at output's mangle priority — only a new connection is classed, the boxes subtree takes the cohort bit and the rest of the slice the node bit, the second guarded by the mask so the boxes rule's mark is final — and postrouting now translates by the mark, so all four chains load in the one transaction and the identities never exist without their classification. The two bits default to 0x30000000 (above Tailscale's 0x00ff0000 and kube-proxy's 0x4000/0x8000); --ct-mark-mask overrides them with exactly two contiguous bits, both rules write the mask's bits alone so nobody else's survive, the ruleset the host already carries is scanned for a ct-mark user of the chosen bits (nft list ruleset, plus iptables-save where it exists) and a conflict is refused naming the rule and the override, and the chosen mask is recorded beside the presence marker so uninstall, --check and the daemon's probe read one value. The install harness pins the classify chain, the mark-keyed postrouting and its lack of a socket match, the mask argument's refusals, and the record. The daemon's probe now requires the recorded mask beside the marker — a marker without it is a step that did not finish — and the ruleset tests pin the classify chain the renderer now emits.
Four proofs in classifier_root_integration.rs, joining the two refusal proofs round 9 left: - classifier_table_loads_on_this_kernel lists the loaded table back and pins the classify chain at output's mangle priority, the two guarded ct-mark rules (boxes subtree → cohort bit, rest of the slice → node bit, both on ct state new and writing the mask's bits alone), and a postrouting chain that translates by the mark and carries no socket match — the shape the kernel refused outright at postrouting, which is the regression this proof exists to catch. Pins read the kernel's own listing, so they match on quoted paths, values and verdicts rather than nft's operator spelling. - snat_identity_is_seen_by_the_peer drives real connections over a veth to a listener in a network namespace the proof holds, so the far end reads the identities NET-078 names: a box leaf's connection arrives as the cohort address, the daemon leaf's as the node plane's, a process outside the tree keeps the host's own source, and loopback is never translated. The observer is this binary re-exec'd inside the namespace, reporting each accepted connection's source to a file the proof reads bounded; it passes on its own, with nothing to observe. - foreign_ct_mark_bits_survive_classification loads a witness table that classes connections with two foreign bits below the mask before the classify chain runs, and reads the marks back with counters after it: a box flow carries the cohort bit over intact foreign bits, a slice flow the node plane's bit over intact foreign bits, and the counter for any flow that lost a foreign bit stays at zero. - install_refuses_an_overlapping_ct_mark_user proves the scan live over a standing ct-mark user: the install refuses, naming the rule it found and the --ct-mark-mask override, and lays out nothing; told to class with non-overlapping bits it goes through and records the mask it chose beside the marker. The proofs run as the lane runs them — unprivileged, privileged steps through sudo -n — and are #[ignore]d like the two before, declined only on a host with no nft. ScratchInstall now removes the ct-mark mask records by prefix with the tree, and net/classifier.rs takes the rustfmt pass its last change missed.
…ost-address-boxes-apart-from-node-plan
|
Round 10 of 10 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. The installer's table now loads whole on the kernel that refused it in round 10's lane: the classify chain runs at output's mangle priority, writes only the mask's two ct-mark bits on a new connection (boxes subtree to the cohort bit, the rest of the slice to the node bit behind a mask guard), and postrouting translates by that mark alone with no socket match, so a peer behind a veth reads the cohort address from a boxes-leaf connection, the node-plane address from a slice-leaf one, the host's own source from a process outside the tree, and nothing on loopback; foreign ct-mark bits another component classes with survive classification untouched; the install refuses over a host already using the chosen bits, naming the conflicting rule and the --ct-mark-mask override, records the mask beside the marker for the daemon's probe and uninstall to read, and the daemon's probe now requires that record to say the step is installed. Notes from the agent: Round 10's four root-lane proofs are written but could not be executed on this box, and no PASS line for them can be quoted from here: this runner has no nft, no sudo, and no cgroup2 mount with nsdelegate (all three checked this session; uid 1000). classifier_table_loads_on_this_kernel, snat_identity_is_seen_by_the_peer, foreign_ct_mark_bits_survive_classification and install_refuses_an_overlapping_ct_mark_user, and round 9's two proofs with them, are owed by the CI minimald-root-integration job, which runs this binary with --run-ignored all --no-tests=fail under sudo; each proof declines, with the reason printed, only on a runner with no nft, and fails on every other precondition it names. What could run here did: the four nextest lines above, the installer's shell harness, shellcheck, and clippy. Two design notes a reader of the lane output will want: snat_peer_observes_the_source is the identity proof's observer half — it passes standalone by declining (nothing drove it), and only snat_identity_is_seen_by_the_…[truncated] |
- classifier.rs: raise PROBE_DEADLINE from 1 s to 3 s; a rejected SYN fails connect() only at its first retransmit (the first ICMP is a soft error) - classifier.rs: decide_names_the_cause_... removes the mask record before re-creating the marker, so "marker without record" is actually set up - session_host/tests.rs: the probe-cause fixture writes the ct-mark mask record beside the marker, as the installer does - install-host-classifier.sh: check nft and scan for ct-mark conflicts before laying out the tree, so a refused install touches nothing - install-host-classifier.sh: replace `[ ] && [ ] || die` with if (SC2015) - classifier_root_integration.rs: pin nft's folded set masks (0xdfffffff / 0xefffffff) and check rules per line, not a glued chain - classifier_root_integration.rs: witness set_foreign chain at priority -180, after conntrack at -200 - classifier_root_integration.rs: accept nft's `|` rendering in the refusal, escape with 0x0c000000 (clear of Tailscale, kube, the default)
|
Approved: no findings at Nothing to do. Verdict APPROVE, posted as a comment: the App cannot review its own pull request; the check and the label carry it. Clobber: none: |
The task's one shared reader had raised both lanes' not-refused probe record to warn, which changed what T48 (#1805) had chosen for the native lane: the reading over a table a person installed, carried at info so the launch's own record beside it is what alarms. The two lanes keep their own levels now, threaded the way decide() already threads the lane itself: the guest's reading is its boot's own product, and a launch that reads anything but a refusal is the one line a bundle carries of why that guest refuses every deny-all host-address box, so it stays a warn there; the native lane is T48's and is not changed here, so it reads at info again.
…VM host decides a host-address box's verdict per box (gominimal#1886) * feat(minimald): the coder's uncommitted work as it stood when its 7200s ran out the coder's uncommitted work as it stood when its 7200s ran out The coder was stopped at the end of its time with these changes not committed. The runtime committed its working tree as it stood so the work is kept: this commit is a snapshot, not a finished step, and may not build or pass its tests. * test(minimald): give the guest's decision a listed table in the cause test The guest's recheck reads its table back out of its kernel with nft list, so a guest that decides per box is decided over a listing a loaded table answers with — the decision test's guest arm now hands it that fact instead of reading a test host's missing /usr/sbin/nft, which read the marker's guest as not effective and failed the suite. Also collapse the three nested ifs clippy refused in the guest's render and load, the same code with one fewer branch. * test(minimald): pin the guest's classifier load and per-box decision The guest half of NET-079 — the boot's render, check, load and marker, the recheck behind the marker, the effect probe over held listeners, and the per-launch decision the network plan follows — is now pinned by the tests the task names: - the guest's render is the installer's own, byte for byte, over both spellings of the two identities (the collapsed pair an un-enrolled guest has, and the harness pair), and the digest of a load covers exactly the bytes piped to nft, in the guest's lane and the install's, over one render, never a re-render; - the installer runs from the one compiled-in file on stdin, with its environment cleared and every parameter as argv after --print-ruleset, never interpolated into script text; - the marker is the load's commit point: written only after the check and the load, cleared before either, absent on any failure, and a failed load leaves the guest refusing deny-all boxes again on its own cause, which names no install command inside a microVM; - the decision re-reads the table behind its marker with nft list before anything rests on it, and only a refusal the chain read as decides per box — a reading that could not run, or a probe whose listener is not live, reads as unknown and refuses the deny-all box rather than guess either way; - the dstnat carve-out is the deny subtree's alone: one rule, the answerer's address and port, never loopback-wide :53 and never the node plane's DNS; - a deny-all host-address box on a VM host resolves through the guest's own answerer when its launch decided per box, and falls back to the node's DNS layer when it did not — the memo the launch's reader recorded, read per launch. Also fixes the clippy lane the CI check names: three let-chains where the new guest code nested ifs, and the decision test the guest's recheck made read a table no test host can list. * test(minvmd): prove the guest's loaded classifier table on a vm host The vmhost layer the task's third round owes: eight guest_kernel / guest_classifier proofs, one VM boot each through the supervisor path, each asserting what the layer states — that the guest's nft -c accepted the rendered table and the load line carries the digest of exactly the bytes piped to nft -f -; that the digest equals the sha256 of the native installer's --print-ruleset output for the same parameters, so the guest and the native installer load one ruleset; that the deny-all box launches over the daemon's own nft-list recheck; that the box's own outbound probe reads EHOSTUNREACH; that its own box name resolves through 127.0.0.1:7656, the one carve-out, with the answerer's serving and answered-lookup lines tying the reply to that port; that a connection another box opens to it through the hostname proxy is answered, the reply-direction admission; that the launch's own record says host_ip_enforcement=per_box; and that every effect probe the guest ran read a refusal per family with an errno inside EHOSTUNREACH, EACCES and EPERM, named in the probe's own record. All eight are gated behind minvmd_libkrun plus MINVMD_E2E=1 with MINVMD_VM_LANE turning a missing precondition into a lane failure, the arm egress_allowlist defines; on this box, where libkrun is absent, the file type-checks only under the forced cfg. * fix(minimald): log every half of the guest's failed table load The guest's boot load dropped two of its four failure variants on the floor: a render that could not run and a marker that could not come or be written returned from load_guest_table with no line of their own, and the boot's caller drops the Result outright, so a guest whose load stopped before its check — a bash that is not there, a marker a daemon death left undeletable, a marker that could not be written over a loaded table — said nothing at all while reporting no per-box verdict. Every variant now logs at error level at its own failure site, the same discipline the check and load lines already keep, each naming its own consequence: the stale marker that stopped the load before anything rendered, the render that ran no check and wrote no marker, and the marker that could not vouch for a table that loaded. The proof is the render half: a load through a bash path that does not exist returns Render with the spawn's own words naming it, runs no check behind it, leaves no marker — not even the one a previous load wrote — and puts the error line on the daemon's log. * fix(minimald): carry the launch's own decision into its network plan The guest's network plan read the verdict decision through a process-wide memo: decide_now wrote it, plan() read it back, and the awaits between the two let a concurrent launch's reading stand in for the launch's own - a deny-all host-address box could resolve through the answerer on a decision another launch read, and a task's host-address plan could follow a session's. network_for now takes the decision as a parameter, the launch passes the value its own reader read, and task_network passes None on the ground that a task places no classifier leaf. The memo (READ_DECISION, note_decision, freshest_decision) is deleted; the provider test takes the decision as a parameter across its scenarios, so the two serial attributes that existed only to keep tests off one memo go too. * fix(minimald): bring the guest's loopback up before it holds the probe's listeners The guest's kernel leaves lo down until something brings it up, and nothing in the boot did before the effect probe's listeners were held: the daemon's own egress step, which does, runs after READY. A bind on 127.0.0.1 over a loopback that is not up is EADDRNOTAVAIL — the error the macOS guest logged at boot, and with it a daemon holding no V4 listener that reads its loaded table's effect as unknown and refuses every deny-all host-address box for a cause its own boot could have ended. Bring lo up first, as the first step of the guest's classifier boot, now composed in one place so the order is the thing a test drives rather than what it assumes: loopback up, then the listeners held for the daemon's life, then the table load. Each half is still the boot's to attempt whatever the one before it did — a loopback that will not come up, and listeners that will not bind, are logged and the load runs behind them, because the marker's table is still the decision's own fact. The bind keeps its tolerance for a family this host does not carry. * fix(minimald): bound the guest's bash and nft runs and kill what outlives the bound The guest's render, check and load ran in pid 1 before READY over waits with no deadline: a bash or an nft that never returned would stop the daemon from ever serving, with nobody watching to interrupt it. Every run is now held to one bound, the boot's own GUEST_CHILD_DEADLINE, generous for runs that take milliseconds each. The feed is part of the run, so the script and the ruleset are piped from a thread beside the wait — a child that never reads cannot hold the write past the bound — and past it the child is killed, reaped so pid 1 carries no zombie, and the half it was running fails as its own GuestLoadFailure, logged like any other with the bound it overran. No marker is written for a table nobody loaded, and the previous table is untouched: the batch never commits. The check and load lines no longer say refused, because a run that never answered did not refuse anything; the VM lane's grep for the check's failure line moves with the wording. The hook runner's own bound keeps the same caveat this one now documents: the bound covers the child the boot spawned, not anything that child backgrounded. * test(minimald): rehearse the install lane's delegation to a non-root account The digest test's install lane returned early whenever the suite ran as root, so its digest equality — the one text the guest pipes, the installer's print, and the install's own load all agree on — never ran on the cross or Docker lanes at all. Root had no account to delegate to: the step's sudo seam names the account that ran sudo, and on a root-run lane that account is root itself, the one id the step refuses to delegate its tree to. So the lane now names a non-root id its kernel has a mapping for instead, read from the namespace's own uid and gid maps — a numeric chown to an id no namespace maps fails with EINVAL, so the map, not a guess, decides — with nobody's id standing in when a map carries no non-root one. The derivation is pinned by a test that runs on every lane, root or not, and the install lane itself no longer skips, so the digest it asserts is asserted everywhere the suite runs. * revert(minimald): give the native effect reading back its T48 log level The task's one shared reader had raised both lanes' not-refused probe record to warn, which changed what T48 (gominimal#1805) had chosen for the native lane: the reading over a table a person installed, carried at info so the launch's own record beside it is what alarms. The two lanes keep their own levels now, threaded the way decide() already threads the lane itself: the guest's reading is its boot's own product, and a launch that reads anything but a refusal is the one line a bundle carries of why that guest refuses every deny-all host-address box, so it stays a warn there; the native lane is T48's and is not changed here, so it reads at info again. * chore(minimald): drop the deleted decision memo from the serial_test comment * fix(minimald): serialize the effect probes that share the daemon's one probe leaf Two host-address launches read concurrently in one daemon, and both probes used the pid-named leaf boxes/deny/filter-probe-<pid>. The second met the first's leaf as EEXIST, read as unreadable, and the guest refused its deny-all box. A process-wide lock now holds the leaf for one probe at a time. * test(minvmd): keep the proxy client's python indentation in the deny-all inbound test The client script was built from a backslash-continued Rust literal, which strips each continued line's leading whitespace, so the box ran an unindented script that python rejects before connecting. The server then waited out the exec timeout. Join the script from lines that keep their indentation, bound the accept and the retry loop at 20s inside the exec budget, and assert the client's result first so a failed request is named by what it met. * fix(minimald): render no resolver carve-out on a VM host The VM-backed guest's node DNS layer applies no per-box name rule to host-address boxes, so the guest's table carves no resolver out. The installer gains --no-resolver-carve-out, which omits the answerer admission in deny_out and the dstnat chain; the guest renders with it and logs one line naming the follow-up (gominimal/inbox#897). A deny-all host-address box in a deciding guest gets an empty resolver, and its DNS to the gateway is refused like any other destination. The native render is unchanged. * test(minvmd): prove a deny-all box's dns to the gateway is refused on a vm host Replace guest_deny_all_resolver_reachable with guest_deny_all_dns_to_gateway_refused: a deny-all box's udp and tcp 53 to the gateway reads an errno in the reject set, and the guest's render logs that it carves no resolver out. The native digest comparison renders with the guest's own parameters, --no-resolver-carve-out included. * fix(minimald): refuse a native deny-all box whose carve-out is not the live answerer The installer records the resolver carve-out it loaded beside the marker (carve-out-<address>-<port>) and clears it with the marker, as it does the ct-mark mask. The native decision reads that record, and server.rs keeps a process-wide cell with the address and port the zone answerer actually bound, cleared when its receive loop exits. A native deny-all launch over a deciding table is refused as TableNotEffective "stale carve-out", naming both values, when the record differs from the live bind, no answerer is bound, or no carve-out was recorded; the hint re-renders the install with --answerer-address/--answerer-port set to the live bind. The box's resolver is the live bind's address. A box that is not deny-all is unaffected. * test(minvmd): accept no reply as the refusal of a deny-all box's udp dns A udp send to a refused destination often succeeds locally, so the udp leg asserts no reply arrives within the bound; the reject's errno or ECONNREFUSED on the connected socket also passes. The deny chain rejects, so the tcp leg keeps the reject-set errno. * test(minvmd): launch the deny-all box before reading its classifier line The per-launch classifier line is written when the box's leaf is placed at its first launch, not at create. Run one command in the box, then read the line within the log deadline. * test(minvmd): strip ansi colour from the guest console before matching it The daemon's tracing output is coloured, so a structured field reads with escape codes between the key and its value and a plain key=value needle never matches. * fix(minimald): bound the guest's per-launch nft list recheck The recheck runs before each host-address launch with no deadline, so an nft wedged on its netlink read would hold the launch forever. Run it through the bounded child wait every other guest child uses; past five seconds the table reads as gone. * fix(minimald): probe a guest launch over the boot-held listener The per-launch reading bound a fresh loopback listener on a guest as on a native host, so the listener the guest's boot holds for its effect probe was used only at start. A guest launch now reads over the held listener; a native host still binds its own per reading. --------- Co-authored-by: minimal-foundry[bot] <327719912+minimal-foundry[bot]@users.noreply.github.com>
T48 — Classify host-address boxes apart from node-plane traffic and enforce their deny-all
Plan block
Proof
A host-address box declared deny-all opens nothing except a connection to its resolver's address and port, and it cannot move out of its cgroup. On a host that cannot decide per box, session start records the cause, never prompts, and runs the box unenforced. The advisory's create-response and CLI surfaces are T73's (#1806).
The table
The privileged step loads one
inettable in a singlenft -ftransaction. The transaction replaces any previous table atomically, and the marker is written only after the load succeeds.outputsocket cgroupv2onboxes/denyat that subtree's own level and jumps todeny_out.deny_outreject with icmpx admin-prohibited, an active refusal, never a silent drop.dstnatpostroutingboxessubtree to the cohort address and the rest of the slice to the node-plane address (NET-078). Both rules carryoifname != "lo".Rules are keyed on cgroup path only. Nothing is edited at box launch or stop.
Deciding "can decide per box"
The daemon decides this from the filter's effect, not from the marker (design §7.4), at start and again before every host-address launch:
boxes/denyleaf then connects to a loopback listener the daemon holds, over 127.0.0.1, and over ::1 when IPv6 loopback is enabled.per_box.nonewith its cause.What happens to a box that cannot be enforced
Natively:
TableNotEffective), or whose effect could not be read (ProbeUnreadable), is refused.In the guest:
host_ip_enforcement=none, and every session start says so on the pty and in the daemon log.Hook runs advise nobody.
Changed files
scripts/install-host-classifier.shinettable; DNS DNAT to the answerer; SNAT split with theloguard; required--cohort-addressand--node-plane-address;--print-ruleset; marker written lastcrates/minimald/src/net/classifier.rs(new)crates/sandbox2/src/config.rsClassifierLeafunderboxes/denyorboxes/allow, a constructor per verdictcrates/sandbox2/src/lib.rsbox_leafandcreate_box_leaftake the verdict;install_hint()names both identity flagscrates/minimald/src/net/mod.rscrates/minimald/src/net/provider.rscrates/minimald/src/main.rscrates/minimald/src/session_host.rscrates/minimald/src/session_host/tests.rscrates/minimald/src/nsenter.rscrates/minimald/src/session.rsclassifier_rootandclassifier_mountinfoknobs at the launcher's only production constructorscripts/install_test.sh--print-rulesetparity, the--checkhint flagsCargo.toml and Cargo.lock are unchanged. No packet-filter crate is added.
Tests
cargo nextest run -p minimald node_plane_and_cohort_distinct_sourcescargo nextest run -p minimald host_ip_deny_all_no_outboundcargo nextest run -p minimald host_ip_deny_all_reaches_only_the_answerercargo nextest run -p minimald host_ip_box_resolves_through_answerercargo nextest run -p minimald host_ip_box_cannot_leave_its_cgroupcargo nextest run -p minimald unenforcing_native_host_runs_host_ip_box_unenforcedcargo nextest run -p minimald rendered_ruleset_admits_only_reply_direction_in_deny_subtreecargo nextest run -p minimald deny_all_host_ip_box_answers_the_proxycargo nextest run -p minimald ruleset_digest_covers_bytes_piped_to_nft_in_both_lanesnft -fequal--print-rulesetSupporting tests:
a_marker_survives_its_table_but_decides_nothinganda_listener_the_daemon_cannot_reach_reads_as_inconclusive(probe)which_undecided_host_address_boxes_are_refused_and_with_what_wordsanda_native_launch_over_either_probe_cause_runs_records_none_and_advises(native refusal and advisory)a_guests_placed_unenforced_host_address_box_advises_at_its_start(guest)a_none_mountinfo_knob_answers_over_the_daemons_own_mount_table(live mount table)scripts/install_test.sh(installer)Live table proofs in
crates/minimald/tests/classifier_root_integration.rs, run under sudo by the minimald-root-integration lane:Result: <PASS lines from the lane, or the lane's nft skip line>.
VM lanes:
scripts/session-e2e.shpasses on macOS (e2e) and Linux KVM (test-kvm) at . Both passed at 60f098d, and macOS also passed at 804c036.Verify
min task run agent-verifypassed.Closes #1631
Round 10 verification (final head
728548a5)Root lane
minimald-root-integration, job 111131928541 on728548a59775610c999364f88644c8139fa066a9,15 tests run: 15 passed (1 leaky), 0 skipped:scripts/install_test.sh: 490 passed, 0 failed.Architecture note: the effect probe proves the deny, the identity test proves the SNAT, and nothing loaded the table before round 9. The socket-cgroup SNAT at postrouting was refused by the kernel (
Operation not supported). Round 10 therefore moved classification to an output-hook mangle chain that writes two ct-mark bits, and translates by mark at postrouting. The default bits are0x30000000.--ct-mark-maskoverrides them, and an install over any overlapping ct-mark user is refused. The probe deadline was raised to 3 s once the live table showed that a rejected SYN failsconnect()only at its first retransmit.