Repository navigation
Make sending anything wait until downstream component is ready - #40
Conversation
|
So this turned out to be more of a systemic problem than I thought. Here's a documentation of what I went through. Currently (i.e. w/o this patch), if you do: This would work, but not this: This is because only sending of initial tests for the readiness of the immediate downstream process. My initial patch solves this. But this wouldn't work with the patch: IPs are buffered before Because of how 'Graph' is currently implemented, the above is in reality something this: The interfacing pseudo-ports are set up during I know there must be a less hacky way. I still haven't grasped the entire codebase so it's difficult for me to think at the highest level. Please let me know if you have any idea! Thanks! But for now, this works for all intents and purposes. |
|
See #42 |
Make sending anything wait until downstream component is ready
Checking for downstream readiness was initially a problem as described in noflo#40. That was due to initial packets being sent before the network is ready. It's partially solved in noflo#69 when initial packets are only sent upon the respective network has been loaded. This PR solves the other half which is initial packets being sent in order of the programmer's specification so by the time the network is run, everything should be ready anyway.
…work documents #40, #45) The remote-runtime scenario for the boundary-owned end-of-transmission contract: a client can now drive a stream into an exported inport over the protocol — hold the boundary open across a packet sequence with back pressure respected, then signal EOT and let the drain-based closing cascade end the network. - Engine: Network#addDriver attaches a persistent external driver socket to a resolved inport (the IIP path minus send/disconnect); the connection lifecycle of boundary sockets (no in-graph sender — driver-attached and IIP) is emitted as a network connection event and recorded in flowtraces as 0x0b CONNECTION_OPEN / 0x0c CONNECTION_CLOSE with the receiving side identified and the source nil - Protocol: the 0x49/0x4A opcodes in the execution block with codec round-trips, the AsyncAPI channels, and the SPEC §7/§8 entries; the tracefile decoder handles the boundary (nil-source) connection payload form - Runtime: the stream registry keyed per plane and port — 0x49 attaches and connects the driver, 0x47 routes through an open driver (the IIP single-packet path otherwise unchanged), 0x4A disconnects and deletes; duplicate opens and stray EOTs answer 0x17 REASON.INVALID_STATE. The stream lifecycle records as 0x0b/0x0c telemetry events Without an open stream the wire behavior is unchanged: each 0x47 injection remains a self-contained single-packet stream. The stream lifecycle is pinned by specs in all three packages: the engine's driver-held-open test, the codec round-trips, and the runtime's open → packets → EOT → network-end sequence with its rejections.
Declarable framing termination, stranded-framing diagnostics, migration-guide contracts (#40)
Sending no longer ends the transmission: InternalSocket.post auto-connects but never auto-disconnects, and the connection stays open after the send. disconnect() is the sender's explicit decision that nothing more will come from it — the 0.x intent, per work document noflo#40 update noflo#5 — which per-burst framing could never express. The same-tick auto-disconnect was a 1.x timing-compat shim; with every component on the Process API and every graph redesigned for 2.x, the framing it provided is redundant with in-band brackets. Exploratory (stream-termination): the suite is knowingly red on tests whose finalization rode the outport's last auto-disconnect — ordered bracket forwarding, subgraph bracket forwarding, scope isolation, NetworkLifecycle merging. The replacement is EOF propagation (input disconnect = final activation = output finish), landing in the next commits.
InternalSocket gains terminal-close semantics: disconnect() marks the socket terminally closed (idempotent, emitted even when never connected, so a silent sender still EOFs its downstream), and connect() re-opens a terminal socket — connect is the beginning of transmission. InPort tracks end-of-stream per port, unscoped by definition: EOF means "there will be nothing more coming from this sender", so a component closing down puts every attached edge at EOF in every scope at once. The eof lifecycle event fires on both transitions — entering EOF and a new transmission re-opening the input. Component derives the classical cascading closedown from it: when every input is at EOF, no activation is open, and no buffered deliverable data remains (control ports excluded — standing values are legitimate at quiescence), the component terminally disconnects its outports, and the closedown event fires only after those disconnects have drained. An input re-opening revives a closed-down component. Self-driven components declare generator: true — input EOF must not close down a component that activates from its own clock. Subgraph wrappers close down with their inner networks: the shared-port EOF reflects the deepest sender's state, not the wrapper's completion, so the wrapper's closedown is driven by the inner network's end. Exploratory (stream-termination, work documents noflo#40 update noflo#5, noflo#45).
…s brackets at arrival
An explicitly-declared empty forwarding rule — c.forwardBrackets =
{ msg: [] } — terminates bracket framing on that port: brackets are
consumed at arrival, they never fire the process, never buffer, and no
output context is mirrored. The component reads pure data.
This is the declarable fix for per-packet readers of bracketed streams,
which otherwise strand the trailing closeBracket: getData silently eats
the leading brackets, and the trailing one arrives after the last data,
matches no data-read pattern, and is declined forever. The declaration
replaces what no amount of pattern analysis can infer — the component
states its intent, and the engine can act on it soundly. The built-in
default's silent adaptation (a default rule whose outports do not exist
is deleted) is unchanged; only an explicitly-declared empty rule gains
the termination semantics.
Work document noflo#40 update noflo#13
…last sender disconnects When an input port's last attached socket terminally disconnects, any bracket framing still buffered on the port — in any scope or index — can never pair with arriving data: the future is known. The residue is surfaced at that moment, per port and per scope: a debug-level log naming the component, and a 0x0f STRANDED event on the network event bus and in Flowtrace (EVENT_TYPE.STRANDED, payload [node, port, scope|null, open, close]). The residue is reported, never auto-consumed: a component may legitimately decline a closeBracket now and consume it after its other input arrives, so silent reaping is unsound — the loudness is the contract enforcement counterpart. This moves the discovery of the stranded-bracket class from the shutdown buffer dump to the moment it became inevitable, and gives the flowtrace artifact the evidence (work document noflo#40 update noflo#13). fbp-protocol pins the 0x0f event type and its payload rule (SPEC); the tracefile round-trip spec covers the flat tuple passing through the generic payload path
This is handy when you're doing something like:
where
SomeComponentmay not be ready and the initial is already sent toMergeknowing nothing about the downstream components.