Repository navigation
Switch to cheerio for scraping. - #28
Merged
Merged
Conversation
bergie
added a commit
that referenced
this pull request
May 23, 2012
Switch to cheerio for scraping.
bergie
added a commit
that referenced
this pull request
May 23, 2012
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Work document #28 (proposed): the isomorphic FBP Protocol 2.0 runtime for NoFlo over Reticulum, superseding the noflo-runtime-base / noflo-runtime-websocket lineage — Reticulum-only, protocol-2.0-only, capabilities as the 0x02 mask instead of protocol: strings. Isomorphic per the no-build browser contract (work document #18): plain ESM, no Node builtins in the runtime core; the application supplies its own Reticulum instance with platform-appropriate interfaces (TCP on servers, WebSocket/WebRTC in browsers), matching @reticulum/core's subpath-import design. Dependencies: the 2.x engine (@noflo/noflo, @noflo/graph), the wire format (@noflo/fbp-protocol), the transport (@reticulum/core), and @reticulum/lxmf for the offline telemetry drops (wired behind a subpath export when telemetry lands). The smoke test pins that the entrypoint and its full dependency graph import cleanly.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 1 of work document #28: RuntimeServer, the frame-routing heart the Reticulum binding feeds. It decodes inbound frames through @noflo/fbp-protocol's decodeFrame dispatch, hands typed decoded frames to handlers registered per command code, and emits outgoing frames through injected send/broadcast functions — the core never inspects contexts (over Reticulum, a context is the link a frame arrived on) and constructs no transport, keeping it isomorphic. - capabilitiesMask composes the 0x02 capability mask from names, failing loudly on typos; the default advertises the read surface (graph, metadata, telemetry, components read-only) - authorize() sends the auth response unilaterally — the transport calls it on link_established with a verified identity - undecodable frames (garbage, unknown opcodes, malformed payloads) and handler failures never throw out of handleFrame: they surface as undecodable / error events, since the protocol has no error channel for them and a runtime must survive hostile link traffic; commands with no handler yet emit unhandledframe
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 2 of work document #28: RegistryProtocol, the Two-Step Cache server side, driven end-to-end through the runtime core in tests. The runtime consumes an application-supplied catalog — the same philosophy as the engine's application-supplied component registry (work documents #6/#16): signatures are ready at handoff, and the handler derives the wire state from them. A signature is exactly the 0x24 detail shape (declared kind plus the full port field set), valid with no implementation behind it, so manifest/sidecar-sourced stubs flow through unmodified. - 0x20 syncs answer 0x21 on hash match, 0x22 manifest otherwise — the runtime always answers (work document #4 update #18) - 0x23 details answer with only the requested known components; unknown names are omitted, and an empty response is an answer - 0x25 source writes and 0x27 installs map onto optional catalog hooks (source and signature are orthogonal, so a write never relabels a component); missing hooks surface as unsupported events - every catalog change re-derives sig_hash values and the registry_hash through the protocol package's canonical serializations Supporting changes: the runtime core now accepts async handlers (WebCrypto hashing is async), capturing rejections as error events like synchronous throws; @noflo/fbp-protocol re-exports its ComponentDetail/PortInfo/ ManifestEntry typedefs from the entrypoint so dependents can type against them.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 3 of work document #28: GraphProtocol, the 0x10/0x14 server side, with the live GraphModel as the epoch. - The epoch id is content-addressed: SHA-256 hex over the graph model's canonical serialization, computed lazily and cached until the graph changes — epoch comparison needs no clocks to be meaningful - The sync handshake answers 0x11 on epoch match, and 0x12 with the new epoch plus a baseline resource hash otherwise; the baseline bytes come from an injected resource provider the transport backs (missing provider surfaces as an unsupported event, since stale clients cannot resynchronize without one) - Inbound 0x14 ops apply through the model's mutators and rebroadcast to every client except the sender, converging the op log; echo suppression keeps the model's own events from re-emitting applied operations; tombstones are idempotent; dropped operations (metadata for unknown entities) do not propagate, because the runtime is the convergence point - Outbound: model add/remove/change events map to the total op projection — inserts carry the entity definition without its id, IIPs unwrap from.data to the wire's data field, removals become tombstones, metadata changes become UI-metadata ops, graph-level metadata changes address nil entity_id — each stamped with the runtime's client id and an incrementing logical clock - Node renames surface as unsupported events: they project at the changeset→protocol boundary (work document #4 §5), not in the runtime The runtime core's broadcast now takes an exclusion context so convergence broadcasts can skip the originating link.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 4 of work document #28: the 0x30/0x32 streaming telemetry surface, fed by a real NoFlo network. NetworkHost owns the network lifecycle: builds the NoFlo network from the runtime's graph with an injected component loader (platform-neutral, per work documents #6/#16), re-emits the network's EventTarget events so protocol handlers observe a network before, after, and across network recreations, and supports rebuild-on-discard. The engine gap it surfaces is documented: the 2.x engine consumes the graph at connect() and does not follow live graph changes, so CRDT operations applied while a network runs update the wire state but not the running network until the engine learns to follow graph deltas. TelemetryProtocol maps host events onto the 0x32 vocabulary: - data packets become DATA events with raw payloads; brackets become BEGIN_GROUP/END_GROUP with the group name - network start/stop become LIFECYCLE START/STOP codes - process errors become ERROR events with the exception string — except errors in the stub error class (default StubNotImplementedError), which become STUB_ERROR: the event type itself is the classification (#4 update #9) - payloads the wire cannot carry are frugalized to type-named placeholders so serialization can never take down the flush loop Flush policy is the runtime's physical decision: per-subscription buffers flush on timers at the effective cadence (the client's requested interval, bounded below by a configurable floor), with explicit flushing available for transports that prefer it. The protocol has no unsubscribe command; subscriptions live with their link, and dropContext tears them down on link close. The integration test streams a real network run — a real ComponentLoader, asComponent factory, graph with IIP — through the full pipeline to codec round-trips. @noflo/noflo re-exports the Network class, restoring 1.x surface parity that the host needed to type against.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Work document #28 (proposed): the isomorphic FBP Protocol 2.0 runtime for NoFlo over Reticulum, superseding the noflo-runtime-base / noflo-runtime-websocket lineage — Reticulum-only, protocol-2.0-only, capabilities as the 0x02 mask instead of protocol: strings. Isomorphic per the no-build browser contract (work document #18): plain ESM, no Node builtins in the runtime core; the application supplies its own Reticulum instance with platform-appropriate interfaces (TCP on servers, WebSocket/WebRTC in browsers), matching @reticulum/core's subpath-import design. Dependencies: the 2.x engine (@noflo/noflo, @noflo/graph), the wire format (@noflo/fbp-protocol), the transport (@reticulum/core), and @reticulum/lxmf for the offline telemetry drops (wired behind a subpath export when telemetry lands). The smoke test pins that the entrypoint and its full dependency graph import cleanly.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 1 of work document #28: RuntimeServer, the frame-routing heart the Reticulum binding feeds. It decodes inbound frames through @noflo/fbp-protocol's decodeFrame dispatch, hands typed decoded frames to handlers registered per command code, and emits outgoing frames through injected send/broadcast functions — the core never inspects contexts (over Reticulum, a context is the link a frame arrived on) and constructs no transport, keeping it isomorphic. - capabilitiesMask composes the 0x02 capability mask from names, failing loudly on typos; the default advertises the read surface (graph, metadata, telemetry, components read-only) - authorize() sends the auth response unilaterally — the transport calls it on link_established with a verified identity - undecodable frames (garbage, unknown opcodes, malformed payloads) and handler failures never throw out of handleFrame: they surface as undecodable / error events, since the protocol has no error channel for them and a runtime must survive hostile link traffic; commands with no handler yet emit unhandledframe
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 2 of work document #28: RegistryProtocol, the Two-Step Cache server side, driven end-to-end through the runtime core in tests. The runtime consumes an application-supplied catalog — the same philosophy as the engine's application-supplied component registry (work documents #6/#16): signatures are ready at handoff, and the handler derives the wire state from them. A signature is exactly the 0x24 detail shape (declared kind plus the full port field set), valid with no implementation behind it, so manifest/sidecar-sourced stubs flow through unmodified. - 0x20 syncs answer 0x21 on hash match, 0x22 manifest otherwise — the runtime always answers (work document #4 update #18) - 0x23 details answer with only the requested known components; unknown names are omitted, and an empty response is an answer - 0x25 source writes and 0x27 installs map onto optional catalog hooks (source and signature are orthogonal, so a write never relabels a component); missing hooks surface as unsupported events - every catalog change re-derives sig_hash values and the registry_hash through the protocol package's canonical serializations Supporting changes: the runtime core now accepts async handlers (WebCrypto hashing is async), capturing rejections as error events like synchronous throws; @noflo/fbp-protocol re-exports its ComponentDetail/PortInfo/ ManifestEntry typedefs from the entrypoint so dependents can type against them.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 3 of work document #28: GraphProtocol, the 0x10/0x14 server side, with the live GraphModel as the epoch. - The epoch id is content-addressed: SHA-256 hex over the graph model's canonical serialization, computed lazily and cached until the graph changes — epoch comparison needs no clocks to be meaningful - The sync handshake answers 0x11 on epoch match, and 0x12 with the new epoch plus a baseline resource hash otherwise; the baseline bytes come from an injected resource provider the transport backs (missing provider surfaces as an unsupported event, since stale clients cannot resynchronize without one) - Inbound 0x14 ops apply through the model's mutators and rebroadcast to every client except the sender, converging the op log; echo suppression keeps the model's own events from re-emitting applied operations; tombstones are idempotent; dropped operations (metadata for unknown entities) do not propagate, because the runtime is the convergence point - Outbound: model add/remove/change events map to the total op projection — inserts carry the entity definition without its id, IIPs unwrap from.data to the wire's data field, removals become tombstones, metadata changes become UI-metadata ops, graph-level metadata changes address nil entity_id — each stamped with the runtime's client id and an incrementing logical clock - Node renames surface as unsupported events: they project at the changeset→protocol boundary (work document #4 §5), not in the runtime The runtime core's broadcast now takes an exclusion context so convergence broadcasts can skip the originating link.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 4 of work document #28: the 0x30/0x32 streaming telemetry surface, fed by a real NoFlo network. NetworkHost owns the network lifecycle: builds the NoFlo network from the runtime's graph with an injected component loader (platform-neutral, per work documents #6/#16), re-emits the network's EventTarget events so protocol handlers observe a network before, after, and across network recreations, and supports rebuild-on-discard. The engine gap it surfaces is documented: the 2.x engine consumes the graph at connect() and does not follow live graph changes, so CRDT operations applied while a network runs update the wire state but not the running network until the engine learns to follow graph deltas. TelemetryProtocol maps host events onto the 0x32 vocabulary: - data packets become DATA events with raw payloads; brackets become BEGIN_GROUP/END_GROUP with the group name - network start/stop become LIFECYCLE START/STOP codes - process errors become ERROR events with the exception string — except errors in the stub error class (default StubNotImplementedError), which become STUB_ERROR: the event type itself is the classification (#4 update #9) - payloads the wire cannot carry are frugalized to type-named placeholders so serialization can never take down the flush loop Flush policy is the runtime's physical decision: per-subscription buffers flush on timers at the effective cadence (the client's requested interval, bounded below by a configurable floor), with explicit flushing available for transports that prefer it. The protocol has no unsubscribe command; subscriptions live with their link, and dropContext tears them down on link close. The integration test streams a real network run — a real ComponentLoader, asComponent factory, graph with IIP — through the full pipeline to codec round-trips. @noflo/noflo re-exports the Network class, restoring 1.x surface parity that the host needed to type against.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Runtime verification (work document #28) surfaced a spec gap: the LIFECYCLE_CTRL capability names lifecycle control, but the wire had no command that exercises it — a client could pause a network it could never start. 0x40 CMD_RUN_CTRL gains START = 0x04 and STOP = 0x05: start builds and starts the network, stop stops the running network. Additive enum, no protocol version bump (update #7). A rejected start — component startup failure, graph error — surfaces as a 0x04 FAILED lifecycle event with the detail in an accompanying 0x04 ERROR event, the honest signal work document #4 update #1 pinned. SPEC §8 documents the actions; the AsyncAPI enum grows the values.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 5 of work document #28: the 0x40 block server side, honest about what the engine can do. Implemented for real: - START/STOP drive the network lifecycle through the host — the surface the wire was missing until work document #4 update #20; a rejected start surfaces as a 0x04 FAILED lifecycle event with the detail in an accompanying 0x04 ERROR event (update #1), finally exercised by a real command path - HWM_SET records the network-global backpressure default on the host, applied when the network is next built; the engine resolves the high-water mark at edge construction, so a change while a network runs is deferred with an ERROR flowtrace event saying so Honest about engine gaps (no faked semantics): pause/resume/step, data breakpoints, and per-process disable have no engine primitive — stop() tears down sockets and loses queued packets, violating pause semantics. Those commands surface as unsupported events for the operator and as ERROR flowtrace events for subscribed clients, never as silence or false acknowledgments. Recording them as engine work items is the runtime's verification job. TelemetryProtocol.record is now public so sibling handlers can feed the stream; NetworkHost carries pending high-water-mark configuration applied at build time.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
Phase 6 of work document #28 — the mesh side of the runtime: ReticulumBinding binds a RuntimeServer to an RNS instance the application brings (isomorphic: interfaces are the app's choice — TCP on servers, WebSocket/WebRTC in browsers). It owns the RNS bootstrap: - a single-purpose IN destination announcing under the stable nofo.runtime aspect, its app_data the positional msgpack([protocol_version, destination_hash_raw16, node_name]) of work document #4 §3, so scanners learn compatibility before spending link-establishment airtime and Proof-of-Work - incoming LINKREQUESTs answered via respondToLinkRequest; on each established link, the peer's verified identity (the responder side of link.identify()) triggers the unilateral 0x02 auth response, and every decrypted DATA packet feeds the frame router with the link as context - link closure emits linkclosed so context-bound state (telemetry subscriptions) drops with the link - epoch baselines for 0x12 stale-epoch replies are served through the link REQUEST/RESPONSE API: the resource token is the truncated SHA-256 of the canonical graph bytes — content-addressed like the epoch id — registered as the request path, so a client fetches with link.request(token) using the token the wire carried The one-call assembleRuntime wires the full stack — server, network host, registry, graph sync, telemetry, execution control — from the application's graph, catalog, and optional loader, deriving the registry state before the first sync; bindReticulum attaches the mesh side and connects the baseline provider. Per-link DACAR capability narrowing remains the application's concern; the binding advertises the server's mask. Verified against fake RNS plumbing at the binding's seams (destination factory, respondToLinkRequest, links): announce encoding, the handshake→auth→frames lifecycle, baseline serving, and the full-stack assembly answering real protocol frames. A true two-instance RNS integration test needs an interface with a server side and stays future work.
bergie
added a commit
that referenced
this pull request
Oct 9, 2026
The true two-instance verification (work document #28 phase 6): a real rnsd relay, one process side running the assembled runtime, the other a hand-rolled protocol client. Opt-in via RUNTIME_E2E_RNSD=host:port — skipped otherwise, so CI stays network-free. The client exercises the whole wire story: announce discovery decodes through the §3 app_data codec, Destination.recalled hydrates the peer identity from the announce, the link handshake completes, the client identifies itself and receives the unilateral 0x02 auth response, a 0x20 sync returns the 0x22 manifest, a stale 0x10 returns 0x12 with the content-addressed resource token, and link.request(token) fetches the canonical graph baseline — every frame link-encrypted. What getting it green surfaced, all real transport bugs: - the binding never registered the destination with the transport, so inbound LINKREQUESTs were dropped as not-addressed-to-us — registerDestination is now called (fakes without an RNS instance are tolerated) - the core's send/broadcast were transport-agnostic no-ops: the binding now installs link-aware delivery at start(), tracking live links for broadcast-with-exclusion and wrapping link.send in error-capturing transmit - leaf fallback requires a default interface: the test marks the rnsd uplink with addInterface(iface, true), matching y-reticulum's practice - interfaces must be explicitly connect()ed — addInterface only binds
AsherBond
pushed a commit
to AsherBond/noflo
that referenced
this pull request
Oct 14, 2026
The graph protocol handler is plane-aware: 0x10/0x11/0x12/0x14 carry the plane_id (nil = the running main graph); a sync or op addressed to a plane this runtime is not hosting is answered with 0x17 CMD_OP_REJECTED — the explicit no, never silence — until the multi-plane registry lands (the plane model is specified in work document noflo#4 updates noflo#28-noflo#33; the registry is the staged next step of this wave). RuntimeServer broadcasts runtime-initiated ops with the main plane id, and the 0x02 auth response carries the runtime metadata and the advertised capability ceiling alongside the granted mask, per the external review's runtime:runtime and allCapabilities/capabilities points.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is WAY faster than the previous implementation.