8000 inquisitor: --dst-mac for receivers that only see their own MAC by bluecmd · Pull Request #6 · sonix-network/linkinsane · GitHub
[go: up one dir, main page]

Skip to content

inquisitor: --dst-mac for receivers that only see their own MAC - #6

Draft
bluecmd wants to merge 6 commits into
mainfrom
w124/dst-mac
Draft

bluecmd wants to merge 6 commits into
mainfrom
w124/dst-mac

Conversation

@bluecmd
@bluecmd bluecmd commented Nov 5, 2026 •
Copy link
Copy Markdown
Contributor

Needed to run linkinsane from the SONIX lab's EOS-hosted Alpine VMs for the 202611 release gate (kamelnetworks/sonic-buildimage#124): the hypervisor delivers only frames addressed to the guest's own MAC (plus broadcast) to the test NIC, so the four unicast checks never reached the receiver and forward / ICMP echo failed even on an unfiltered VLAN, while the ASIC counters showed the frames leaving the switch.

--dst-mac <receiver MAC> sends the unicast checks (forward, icmp4 redirect and echo, manymac, ethertype) to that MAC instead of the hard-coded random ones. Without the option nothing changes. The RA check still uses the all-nodes multicast MAC; the lab VMs don't receive IPv6 multicast either, so that check is judged there from the switch's ACL counters.

Driven by sonix-ansible lab-tests/linkinsane.yml, which passes the receiver VM's MAC from the EOS host_vars.

🤖 Generated with Claude Code

https://claude.ai/code/session_012hdRF7EtbB9soNEahorCzg

Second commit: --grace SECONDS adds time to every check's receive window after all checks have sent. On the lab VMs the ICMP checks (0.5 s windows) expired before the receiver's copies arrived through ZMQ, so the redirect check passed and the echo check failed regardless of the switch; --grace 3 fixes that.

bluecmd and others added 5 commits November 5, 2026 08:15
The unicast checks (forward, icmp4, manymac, ethertype) send to random
destination MACs and rely on the receiver sniffing in promiscuous mode.
A receiver whose NIC filters on its own MAC, such as a VM behind a
hypervisor, never sees them. --dst-mac sends them to the given MAC
instead; without it nothing changes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hdRF7EtbB9soNEahorCzg
Every check judges its receive window from when it sent, but the queue
is only drained after all checks have run, and the broadcast check's
tcpreplay burst alone takes longer than the 0.5 s ICMP windows. With a
slow receiver (a VM) or a long out-of-band path, the early checks then
expire on the first dequeued packet without ever seeing their own
copies: the redirect check passes and the echo check fails whatever the
switch did. --grace adds the given seconds to every check after sending.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hdRF7EtbB9soNEahorCzg
ICMPEchoCheck.receive() indexed pkt[ICMP][Raw] on every remote ICMP
packet. The redirect check's own packet ends in the quoted IP/ICMP
header, so once it reached the receiver the inquisitor died with
IndexError before printing the table.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hdRF7EtbB9soNEahorCzg
All checks send back to back and the broadcast check's second burst of
100 frames goes out while manymac and ethertype send theirs. A slow
receiver (a VM, Python scapy sniffer) drops frames under that burst and
the later checks then pass without their frames ever having been seen.
With --sequential each check gets the receiver to itself; its queue is
drained and cleared before the next check starts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hdRF7EtbB9soNEahorCzg
A fixed sleep is not enough: after the broadcast burst a slow receiver
keeps publishing its backlog for several seconds, and the next check's
window then closes before its own frames come through. Drain the queue
until nothing has arrived for the grace period instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hdRF7EtbB9soNEahorCzg
@bluecmd
bluecmd commented Nov 7, 2026
Copy link
Copy Markdown
Contributor Author

Review of exact head b137596a4c5ea20a25d0c7bb8cad90d1e5a5b1bd: --grace 3 does not guarantee the advertised receive window. In the actual while running loop, pkts.get(timeout=1.0) followed by except queue.Empty: break exits on one quiet second even if checks still have time remaining. A blocked-packet check may consequently report PASS before a delayed forwarded copy arrives; positive controls may fail.

Confirmed remotely in sonic-slave-trixie:f5c0481c01d on builder spark2, without network probes or builds. A stdlib fake queue delivers its probe at simulated t=2s; the check deadline is t=3.5s (0.5 + grace3). The test extracts and executes the unchanged while running AST from this exact head:

Actual PR6 loop: elapsed=1.0s, get_calls=1, end_time=3.5s, probe_at=2s, seen=False, remaining_checks=1
Control: same probe delivered at 2s, inside 3.5s window. Early-timeout regression reproduced.

Please continue waiting on an empty queue while a check deadline remains, expire completed windows explicitly, and add a regression for a copy arriving after one quiet second but before the deadline. Also test the no-copy case terminates at the deadline. This is bounded orchestration evidence, not a hardware or full end-to-end test. Evidence scripts are retained at /work/pr6-grace-review/{inquisitor.py,check.py} on sonic-builder-spark2-0. No source changes made in this review.

@bluecmd
bluecmd commented Nov 7, 2026
Copy link
Copy Markdown
Contributor Author

Starting the focused receive-window correction for the confirmed grace regression, as part of kamelnetworks/sonic-buildimage#124. I will preserve the existing protocol and add deterministic delayed-copy, deadline-expiry, and early-completion tests, run remotely without packet transmission.

@bluecmd
bluecmd commented Nov 7, 2026
Copy link
Copy Markdown
Contributor Author

Fixed in descendant commit 7e8410b on this existing PR branch; pushed without force after verifying the previous head was unchanged.

The inquisitor now calls a small dependency-free receive loop that waits through empty queue intervals until each check's own deadline, drops expired checks, and returns immediately when checks complete. Packet construction, wire protocol, result logic, and the sequential backlog drain remain unchanged.

Remote validation in sonic-slave-trixie:f5c0481c01d on spark2: 5/5 tests pass, covering a delayed copy after one quiet second inside the grace window, no-copy deadline expiry, early completion, independent check deadlines, and exclusion of a copy at the deadline. Reintroducing the early queue.Empty break causes three tests to fail. These tests execute the same receive_checks function called by inquisitor; no sockets, packet transmission or builds were used. git diff --check also passes. Run command is documented in README.

The sonix-ansible harness still pins old b137596; it needs an explicit pin update after review. IPv6 RS/redirect probes and EVPN topology coverage remain separate acceptance gaps noted on kamelnetworks/sonic-buildimage#124. No deployment or merge performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0