8000 Phase 1: holepunch mesh (hyperswarm transport + punch URI) — DRAFT by bu5hm4nn · Pull Request #1 · bu5hm4nn/pi-agent-team · GitHub
[go: up one dir, main page]

Skip to content

Phase 1: holepunch mesh (hyperswarm transport + punch URI) — DRAFT - #1

Draft
bu5hm4nn wants to merge 12 commits into
wave-i18nfrom
wave1-hyperswarm
Draft

bu5hm4nn wants to merge 12 commits into
wave-i18nfrom
wave1-hyperswarm

Conversation

@bu5hm4nn
@bu5hm4nn bu5hm4nn commented Oct 3, 2026
Copy link
Copy Markdown
Owner

Phase 1 — Holepunch mesh (draft; for visibility + live two-node test)

A fourth transport: a serverless, NAT-traversing mesh on Holepunch hyperswarm
(DHT discovery, UDP hole punching, Noise per connection). One team per session; a
fresh worker joins by presenting a single punch URI.

Owner UX

  • /team create <name> generates the token + a 32-byte topic, persists them
    (0600) and prints one pasteable punch://<name>/<topic>/<token>.
  • /team join <punch://...> joins from another session with nothing else. Also
    --punch <uri> and TEAM_PUNCH=<uri> pi.

Settled decisions (council review, 2026-10-02)

  • Topic is a generated 32-byte capability stored per team — not derived from
    the room name or the token — base64url in the URI. DHT bootstrap defaults to the
    public set and is injectable.
  • Token gate: an application-level HMAC-SHA256(token, version‖topic‖handshakeHash‖role)
    possession proof sent as the first frame. The raw token is never transmitted;
    the comparison is constant-time; identity and envelopes are gated on prior
    verification; refusal is per-peer (probeAuth-shaped).
  • Host hint dropped: DHT removes seed addresses and the token is the
    authorization, so a third URI field would verify nothing.
  • Dependency: hyperswarm@4.17.2 as an optional dependency, loaded lazily,
    so broker/mesh/swim keep the zero-required-dependency guarantee.

Commits

  • b4037df transport + conformance (01, 03)
  • 9ba504d test hygiene: skip SWIM conformance when the sidecar is absent
  • 04811c4 token possession proof (06)
  • b9b0f8d punch URI (02)
  • 6aee748 one-line join docs (08)
  • 236cd2d hardening (missing-topic refusal, URI-safe tokens, masked tool render)
  • 21dec16 docs (07a)

Validation

  • npm test → 347 tests, 341 pass, 0 fail, 6 skipped (SWIM sidecar absent), exit 0.
  • e2e/conformance.js unchanged. Each wave independently reviewed read-only with
    no blocking findings.

Not yet done

  • Live two-node cross-network check (the backlog requires it for 01/02/06);
    this draft exists partly to enable that.
  • Backlog Status flips and the HANDOFF.md landing note — done at final merge.
  • No merge: merging stays with the owner.

@bu5hm4nn
bu5hm4nn changed the base branch from main to wave-i18n October 3, 2026 14:37
@bu5hm4nn
bu5hm4nn force-pushed the wave1-hyperswarm branch 6 times, most recently from 46ada22 to c101ae3 Compare October 3, 2026 18:46
Story 01 + 03. A fourth transport on Holepunch hyperswarm: nodes join a
32-byte topic, DHT discovers peers, each connection carries the ws.js
framing. from is taken from the connection identity, never a sender's
self-claim. members() is live connections, never self. send() resolves
targets exactly like mesh (@all/@label/names/array) and returns false for
an unknown recipient or empty group.

Per-connection framing is "identity first, envelope second" so story 06's
team-token handshake can slot in before any envelope is accepted. The
topic alone is the capability here.

The native addon is loaded lazily (createRequire inside the factory/start),
so broker/mesh/swim are unaffected. hyperswarm is an optionalDependency,
not a dependency: the zero-required-dependency guarantee stays intact, and
a missing native build fails createTransport closed with an actionable
message instead of breaking other modes.

Tests: e2e/transport-hyperswarm.test.js drives the unchanged
e2e/conformance.js against a local hyperdht testnet bootstrapper injected
through the transport's bootstrap option. Unique random topic per harness,
no public DHT.

Also fixes package.json's test scripts to use glob patterns; Node 24 no
longer accepts bare directories for --test (pre-existing).

Deviation from the literal instruction, per owner decision: hyperswarm in
optionalDependencies rather than dependencies.
Topic is a capability, not a secret: Noise authenticates the peer identity a
node generated, not team membership. Add an application-level HMAC-SHA256
possession proof over the team token so the serverless path authorizes like
broker/mesh/swim do.

- proof = HMAC(token, version || topic || Noise handshakeHash || role), sent as
  the first frame; the raw token never goes on the wire
- both sides send symmetrically, so there is no deadlock
- constant-time verification with the peer's complementary role; a proof cannot
  be replayed in the reverse direction
- identity hello and envelopes are only accepted after the proof verifies
- refusal mirrors probeAuth ({t:'auth_failed', reason}) then destroys the
  connection; auth_failed is per-peer and never latches the room offline
- re-authenticate every connection, bound pre-auth frames, add a handshake
  deadline
- once ready, ignore further identity hello so a peer cannot re-claim a name
- createTransport passes config.token for mode hyperswarm; a missing token
  fails closed with token_missing

Conformance still passes unchanged.
Story 02. `/team create <name>` with no --url and no --mode now defaults to
the hyperswarm mode: it generates the token and a 32-byte DHT topic,
persists both (0600) and prints one pasteable
punch://<name>/<topic>/<token> URI. `/team join <uri>` takes only the URI;
`--punch` and `TEAM_PUNCH` do the same, and the team_join tool accepts a
punch string in place of team + token.

- src/options.js: parsePunchUri/buildPunchUri; `topic` and `punch` keys; a
  strict scheme/field check so a malformed or foreign-scheme string is
  refused with a reason, never silently treated as a team name or seed.
- src/team-config.js: generateTopic + validateTopic; createTeam generates and
  persists the topic for hyperswarm; joinTeam persists a topic from the URI
  and backfills a missing one on old configs.
- src/mode.js: hyperswarm readiness and construction validate the topic is a
  real 32-byte value, not merely truthy.
- src/dispatch.js: create default mode, existing-hyperswarm URI reprint,
  join-from-URI, mode help.
- index.ts: --team-punch/--punch flags, startup TEAM_PUNCH, team_join punch
  parameter, and wizard/help/completion mode lists include hyperswarm.

Tests: parse/build (good + malformed), create->URI->parse round-trip,
join-from-URI persistence, topic persistence + 0600, and the env path.
Story 08. Document `TEAM_PUNCH=<uri> pi` as the outbound-only join for a
rented box; `TEAM_TOKEN` stays an optional override, not a second
requirement, because the URI already carries the token.

Deliberately no `pi-agent-team-join` wrapper: the env-var path reuses the
existing precedence and adds one more value, while a wrapper would add a bin
entry and ongoing maintenance for no new capability. Prefer the env-var path.
Wave 4 hardening for story 02, closing the Wave 3 review findings:

- joinTeam no longer silently generates a fresh random topic for a
  hyperswarm config that lacks one. That put the node in a different DHT
  room than the team, with a silent failure mode (looked joined, nobody
  discoverable). It now refuses and points at /team join <punch URI>;
  /team create is what generates new rooms. The old "backfill topic"
  behavior and its test are gone.
- Tokens are checked against ^[A-Za-z0-9_-]{16,}$ wherever a token is
  accepted (validateOptions, createTeam, parsePunchUri, joinTeam updates).
  This tightens the previous length-only rule in options.js and guarantees
  buildPunchUri always round-trips. generateToken()'s 64-hex still passes.
  OPTION_HELP no longer implies only hex is accepted.
- team_join renderCall masks the token segment of a punch URI instead of
  echoing the full secret into the visible tool-call line.

Tests updated for the new refusals and charset (behavior genuinely
changed). The pre-existing saved:false parameter is left unimplemented.
Story 07a. Extend the existing tables and sections; no new document.

- README: the mode table now covers four transports with
  members-found-by / how-messages-travel / what-it-needs columns; fix the
  "The three modes" heading; document the punch URI as the entire
  per-worker setup with an explicit /team create -> /team join <punch URI>
  flow, and say plainly that the URI carries the token and is the team
  secret.
- docs/how-it-works.md: four-transport table and intro, a Hyperswarm
  mode section in the same style as the other transports, and a
  high-level note on the HMAC token possession proof. Security-model and
  conformance-suite counts updated.
- package.json: description says four transports.

Deliberately not documented here: the delivery digest and the max
message age (story 07b, Phase 2). The "why not followUp" caveat is
unchanged.
Add e2e/hyperswarm-two-node.test.js: a fully self-contained test that
drives two nodes end to end. Node A creates the room and builds a
punch URI; node B joins from that URI alone and its persisted config
matches A's topic/token. Both come online, see each other in members(),
and exchange a ~37 KB multi-byte payload in both directions
byte-for-byte.

Hermetic and CI-ready: depends only on Node and already-declared
packages (hyperdht is a devDependency). A local DHT network binds only
127.0.0.1 with kernel-assigned ports: a bootstrap node plus a routing
node, and each transport is given its own loopback-bound DHT so even
hyperswarm's underlying sockets never bind 0.0.0.0. All traffic stays
on loopback - no public DHT, no internet, no sidecar.

Three details that matter:
- One local DHT node is not enough: hyperswarm's topic announce needs a
  routing quorum, so lookups stay empty. Two nodes discover in ~45ms.
- `new DHT({ port: 0 })` is silently rewritten to the fixed port 49737
  by hyperdht (`opts.port || 49737`); `port: [0, 0]` binds a true
  ephemeral port. The test reads address().port and never hardcodes one.
- The transport's `dht` option lets each node bind loopback, so the
  default 0.0.0.0 listen socket is not used.

Also fix a leaked handle this surfaced: the flush-timeout fallback timer
in start() was never unref'd, so every node kept the event loop alive
for flushTimeoutMs (10s) after start resolved. Unref it, consistent
with the other timers in the transport.

Test file runs in ~0.7s, repeats deterministically, and exits cleanly.
…strings

Route the user-visible strings the Phase 1 work added through t():
transport-hyperswarm diagnostics, the punch-URI reasons in options.js,
the token/topic validation reasons in team-config.js, the create/join
punch-URI output in dispatch.js and index.ts, plus the hyperswarm flag,
menu and autocomplete entries. 36 new keys land in messages.js and both
catalogs in this commit so the per-commit completeness guard holds.

Model-facing team_join descriptions stay Chinese per decision D1.
The Phase 1 punch-URI work replaced the old team_join tool text; drop the
six stale allowlist entries for it and register the eight new model-facing
literal strings that D1 keeps Chinese.
Refine 13 English keys for clarity and actionability: the topic is
generated when the punch URI is created (not by it), name the token and
topic charsets explicitly (A-Z, a-z, 0-9, - and _ only), drop the repeated
'join' / 'copy it directly' calques, capitalize the CLI flag help like its
siblings, and make the empty-module fragment read correctly after the
'failed to load:' wrapper. zh-Hans stays frozen; no other story-19 string
or model-facing Chinese changed.
The rebase conflict resolution left a duplicated 'return completeToken([' in
the index.ts completion handler, making index.ts a SyntaxError so the extension
failed to load. Not caught by npm test because no test imports index.ts.
A rebase left a duplicated import in index.ts, making the whole entry a
SyntaxError; Pi failed to load the extension while npm test stayed green
because nothing imported or parsed index.ts.

Add src/index-loadable.test.js: strips types with node:module's
stripTypeScriptTypes, then runs 'node --check --input-type=module' on the
resulting ESM via stdin and asserts exit 0, surfacing stderr on failure.
A second test asserts the mechanism actually rejects a synthetic bad module.

Note: 'node --check <path>' is a silent no-op on ESM files, so the
file-path form would have given a false pass; the stripped stdin+module
form does the real parse. Verified by temporarily duplicating a line in
index.ts (guard failed with "Identifier ... already been declared") and
reverting byte-identically.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0