8000 fix(minimald,minvmd,switch): on a microVM guest, node-plane traffic leaves as 100.64.255.253 and the by minimal-foundry[bot] · Pull Request #3712 · gominimal/minimal · GitHub
[go: up one dir, main page]

Skip to content

fix(minimald,minvmd,switch): on a microVM guest, node-plane traffic leaves as 100.64.255.253 and the - #3712

Draft
minimal-foundry[bot] wants to merge 6 commits into
mainfrom
agent/3080-on-a-microvm-guest-node-plane-traffic-leaves-as
Draft

minimal-foundry[bot] wants to merge 6 commits into
mainfrom
agent/3080-on-a-microvm-guest-node-plane-traffic-leaves-as

Conversation

@minimal-foundry
Copy link
Copy Markdown
Contributor

Follow-up

Fixes #3080

  • Epic: charged to Box networking — preview by name, bounded egress, and secure reach across hosts #1437
  • Size: M; estimated 180 code, 160 tests
  • May touch:
    • crates/switch/src/lib.rs
    • crates/minimald/src/main.rs
    • crates/minimald/src/guest.rs
    • crates/minimald/src/net/switch.rs
    • crates/minimald/src/net/classifier.rs
    • crates/minvmd/src/box_registry.rs
    • crates/minvmd/src/net/egress_gate.rs
    • crates/minvmd/src/net/bep_stub.rs
    • docs/specs/18-spec-box-networking/18-spec-box-networking.md
Task marker
### D3080 — On a microVM guest, node-plane traffic leaves as 100.64.255.253 and the host-address cohort leaves as its own reserved address 100.64.255.251
  kind: task
  size: M
  est: 180 code, 160 tests
  executor: agent
  origin: residual
  epic: 1437
  layers:
    - switch:
      - crates/switch/src/lib.rs: adds SwitchSubnet::host_ip_cohort_address below the BEP peer, and moves last_ptask below it so no lease takes it
      test: cargo test -p switch host_ip_cohort_address
    - guest:
      - crates/minimald/src/main.rs: passes the cohort address as cohort_address and daemon_ip as node_plane_address to GuestRender
      - crates/minimald/src/guest.rs: adds the cohort address to eth0 beside the daemon address, so SNAT replies return to the guest
      - crates/minimald/src/net/switch.rs: lets the node relay admit frames from the cohort address as well as the daemon address (NET-084)
      - crates/minimald/src/net/classifier.rs: tests that the guest render SNATs the cohort mark and the node-plane mark to two distinct addresses
      test: cargo test -p minimald guest_render_gives_cohort_and_node_plane_distinct_addresses
    - gate:
      - crates/minvmd/src/box_registry.rs: keys the NET-138 cohort row by the cohort address and the node row by the daemon address
      - crates/minvmd/src/net/egress_gate.rs: selects the cohort row or the node row by source address, so node-plane rules never apply to the 
8000
cohort and records name each
      - crates/minvmd/src/net/bep_stub.rs: uses the cohort address as the cohort source in place of host_alias
      test: cargo test -p minvmd gate_decides_cohort_and_node_rows_by_source_address
    - spec:
      - docs/specs/18-spec-box-networking/18-spec-box-networking.md: states the guest cohort address for NET-078 and NET-138, and which row the gate applies to each address
      test: cargo test -p minvmd gate_decides_cohort_and_node_rows_by_source_address
  proof: On a VM-backed host, a flow from a host-address box reaches the gate with source 100.64.255.251 and the cohort row, and a minimald flow arrives as 100.64.255.253 with the node row.

Proof

On a VM-backed host, the guest SNATs a host-address box's connections to 100.64.255.251 and the daemon's own to 100.64.255.253. eth0 carries both addresses. The gate decides a 100.64.255.251 frame by the cohort row (the resident host_ip boxes' rules) and a 100.64.255.253 frame by the node row (the baseline set). The unit tests above show this. No live VM run was made on this box.

Notes from the agent: Files outside the layers, each needed by the switch layer's own change: (1) crates/minimald/src/net/mod.rs tests: the lease run is now one address shorter, so the hand-out and reserve literals move down by one. The /29 cases move to a /28, because a /29 now has a single PTask address. (2) crates/minvmd/src/control.rs test: the same hand-out literal moves. (3) crates/minvmd/src/net/dns_pins.rs: two lines. Host_ip DNS queries and replies were keyed only to the daemon address, so the cohort's name pins broke after the SNAT change. (4) crates/minvmd/tests/guest_classifier_integration.rs (KVM lane, not run here): its native digest now renders with the two distinct addresses that the guest uses. Design choices: the host_ip boxes' compiled rules now hold the cohort address as their source. Because of this, a box-marked frame from the daemon address (an older guest, or a spoof) now drops and is not admitted. Three gate tests changed to send from the cohort address. The node relay in net/switch.rs also rewrites the MAC: cohort frames leave at the cohort address's derived MAC, and frames to that MAC go back to the tap's MAC. Without this, the Box Egress Proxy leg refuses the cohort (it checks the source's switch-derived MAC). Behaviour change: a /29 switch subnet now leaves the daemon's self-allocation reserve empty (one PTask address, which goes to the hand-out run). Pre-existing failures on this box, the same at the base commit: several minimald net::switch relay tests, box_manager/box_actor tests, and minvmd answerer/sock tests (these fail only in a full-process …[truncated]

Tests

line exit tail
cargo test -p switch host_ip_cohort_address 0 running 1 test test tests::host_ip_cohort_address_sits_below_the_proxy_outside_the_lease_run ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 66 filtered out; finished in 0.00s warning: unknown fields in task runtime-implement-test-0: network warning: unknown fields in task runt
cargo test -p minimald guest_render_gives_cohort_and_node_plane_distinct_addresses 0 running 1 test test net::classifier::tests::guest_render_gives_cohort_and_node_plane_distinct_addresses ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1547 filtered out; finished in 0.12s running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 13 filtered
cargo test -p minvmd gate_decides_cohort_and_node_rows_by_source_address 0
cargo test -p minvmd gate_decides_cohort_and_node_rows_by_source_address 0

Verify

min task run agent-verify passed ([tasks.verify]).

The coder's box did not compile these files. Check them in CI: crates/minimald/src/guest.rs, crates/minimald/src/main.rs, crates/minimald/src/net/classifier.rs, crates/minimald/src/net/mod.rs, crates/minimald/src/net/switch.rs, crates/minvmd/tests/guest_classifier_integration.rs.

Base

The base branch moved while the coder worked and was merged in before the tests ran. These files changed on BOTH sides since the branch point — read them first, because a rewrite from a stale reading can undo a merge without a conflict:

  • docs/specs/18-spec-box-networking/18-spec-box-networking.md

Out of scope

These files are outside the task's layers:, so this PR is a draft:

  • crates/minimald/src/net/mod.rs
  • crates/minvmd/src/control.rs
  • crates/minvmd/src/net/dns_pins.rs
  • crates/minvmd/tests/guest_classifier_integration.rs

SwitchSubnet::host_ip_cohort_address sits below the Box Egress Proxy,
and last_ptask now ends below it, so no lease takes it.
The guest render SNATs the cohort mark to the subnet's cohort address
and the node-plane mark to the daemon address. eth0 carries both, and
the node relay admits the cohort address and sends its frames at the
address's derived MAC, which the Box Egress Proxy leg checks.

The lease run is one address shorter, so the allocator tests in
net/mod.rs move their literals and use a /28 for the small-subnet cases.
A frame from 100.64.255.251 is decided by the resident host_ip boxes'
rules, and a frame from 100.64.255.253 by the node baseline. The cohort
record, the host_ip rules, the proxy's cohort source and the DNS pins
key on the cohort address. Hand-out literals in the tests move down by
one with the shorter lease run.
On a VM-backed host the cohort leaves as the cohort address and the
node plane as the daemon address, and the gate picks the cohort's row
or the node's row by the source address.
@coderabbitai
coderabbitai Bot commented Nov 20, 2026
Copy link
Copy Markdown
Contributor

Important

Review skipped

Bot user detected.

Ask an organization administrator to verify that this bot has an active assigned CodeRabbit review seat. See the seat-assignment guide. After the seat is confirmed, request a review again.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ec329f8c-ff92-4831-b7a7-6a4325bbd340

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@github-actions
Copy link
Copy Markdown
Contributor

Spec lint

docs/specs/18-spec-box-networking/18-spec-box-networking.md — 10 finding(s)

docs/specs/18-spec-box-networking/18-spec-box-networking.md:171: P004 NET-009 names test 'native_resolution_without_proxy_env', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:378: P004 NET-040 names test 'fresh_install_own_ip_ingress_publishes_loopback', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:388: P004 NET-041 names test 'installer_switch_binary_executable', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:435: P004 NET-048 names test 'linux_amd64_manifest_ships_vm_stack', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:440: P004 NET-049 names test 'fresh_linux_kvm_activate_local_minvmd', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:445: P004 NET-050 names test 'linux_arm64_manifest_ships_vm_stack', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:450: P004 NET-051 names test 'fresh_arm64_kvm_activate_local_minvmd', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:974: P004 NET-132 names test 'proxy_sees_each_vm_box_by_its_switch_address', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:1000: P004 NET-134 names test 'deny_all_box_reaches_proxy_and_no_other_host_port', which was not found in the repository
docs/specs/18-spec-box-networking/18-spec-box-networking.md:1024: P004 NET-107 names test 'session_outbound_request', which was not found in the repository

Advisory. None of this blocks the merge; the review decides.

Advisory, and never a merge gate. Linter: gominimal/foundry@8acb0663, spec format: How Minimal Writes Specs.

@norrietaylor
Copy link
Copy Markdown
Member

/revise
The last round failed with "A person needs to take it from here" (job b4ad0bb2) and pushed nothing, so this repeats the CI fix. test-kvm and the macOS e2e lane both fail the same test at b90064e, and main is green on both:

crates/minvmd/tests/vm_escape_integration.rs:2122 in vm_escape_bounded_to_resident_union panics with "the gate dropped the admitted marked frame under the host_ip rule; the gate said: … undeclared_box_default=in …".

This PR moves the node plane off the host-address cohort's source address in the guest (#3080). The gate now drops a frame from a resident host_ip box that its rule admits. So the new source or mark for node-plane traffic is likely also being applied to the box's own marked frames, or the gate no longer matches that box's frames to its row. Make the box's admitted frames keep their old source and mark, so only node-plane traffic moves. Then run cargo test -p minvmd --test vm_escape_integration vm_escape_bounded_to_resident_union on a KVM host if you have one. Record the result either way.

@minimal-foundry
Copy link
Copy Markdown
Contributor Author

That /revise was refused. A person needs to take it from here..

Round 2 of 10 was not counted; #3080 carries the same sentence.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

minimald: a VM guest gives the node plane and the host-address cohort one source address (NET-078, NET-138)

1 participant

0