Repository navigation
Wrap groups on the outside - #48
Merged
Merged
Conversation
AsherBond
pushed a commit
to AsherBond/noflo
that referenced
this pull request
Oct 14, 2026
Add a FromAssembly component
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
…ession pin (work document #48) Distills the noflo-jekyll document leg (work document #35 updates #14-#22) to its engine shape and runs it through the actual wrapAssembly machinery: a gate-model render component — standing bracket-framed config on a scoped:false control port read non-consumingly, per-message scoped data on two ports gated on both — wrapped as an assembly adapter (Fork, field extractors, CollectFields), driven with bracket-framed scoped message envelopes whose arrival order races the config and stagger with async jitter inside open activations, then driven to EOT with the closedown cascade asserted. The negative result this suite pins: on the current engine (post-#1094 control gates, #1092 drain-based closing, #1099 drain pump), roughly 1300 randomized runs across staggered, burst, bounded-edge, multi- registration-config, and deep-overlap variants delivered every message and closed down — the intermittent drop class the testbed documented did not reproduce. Three randomized variants run in CI; the full matrix rides the WD48_* env knobs.
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
…closedown (work document #48) Pins the engine mechanism behind the noflo-jekyll intermittent drop (#48 update #2): a gate-model component — standing config on a control port, scoped posts converted through the component itself, unscoped pages held head-of-line until every post converted — driven through the exact interleaving the pipeline races into. All inputs reach EOT while the last consuming activation is open; its completion runs the closedown predicate synchronously in deactivate (buffered data at EOT is 'stranded by definition' per work document #45 and does not block closing), setting closedDown before the pump offer that completion schedules can fire. The pump's guard declines on closedDown, the held resolvable documents never drain, and the shutdown silently discards them. No error, no residue. The test asserts the correct contract — every held resolvable document must emit before closing down — and is RED on the current engine with the diagnostic in the failure message: emitted posts only, closedownFired, held=[r1,r2,r3]. It is the acceptance gate for the engine fix; per-post resume gates make the interleaving exact.
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
…before closing (work document #48) When every input reached EOT while the last consuming activation was still open, the closedown predicate ran synchronously inside deactivate — buffered data at EOT is 'stranded by definition' and does not block closing (#45) — and set closedDown before the pump offer that same deactivation had scheduled could fire. The pump's guard declined on closedDown; held documents that had become resolvable before EOT were silently discarded by the shutdown. The noflo-jekyll testbed r 8000 eproduced the drop end-to-end at ~10% per build (work document #48 update #2). The fix: the closedown evaluation defers while a pump offer is pending — the offer is the one state change that could still drain the buffers — and the offer re-evaluates the closedown after its pass. An offer that consumed leaves open activations whose deactivations re-evaluate; an offer that declined declares the residue stranded with #45's diagnostics. The deferral is buffer-aware: empty buffers do not defer, preserving the closedown-per-burst-cycle granularity. The deterministic pin (spec/wd48-anomaly.js) is green; the randomized stress suite and the whole engine suite pass. End-to-end acceptance: the noflo-jekyll stress went from 2-3 drops per 5 builds to 15/15 deterministic builds (987 files, identical tree).
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
The closedown-vs-pump delivery race: held resolvable documents drain before closing (work document #48)
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
… on their arrivals (work document #48) A forwarding inport's arrival fired the direct path only — one activation for the arriving IP's own scope — while the ready-scope walk that re-fires every other scope holding pending work was skipped. Work whose gate the arrival completes (a corpus arriving on a framing-terminated control port completing a held document's hasStream gate) stranded behind its scope until the shutdown silently discarded it: the noflo-jekyll testbed's intermittent whole-leg drop, localized by per-adapter trace attribution to the liquid Template's includes gate (the corpus arrived, the pages were buffered, the inner never activated). The fix runs the ready-scope walk for the remaining scopes after the direct firing, deduplicating the arriving scope; the forwarding machinery's bracket ownership is unchanged. The one-firing rule's coverage is now complete: every arrival on every port re-fires every scope holding pending work. The deterministic pin (spec/wd48-anomaly.js) reproduces the race exactly — the corpus arrives after the message buffers, the pending scope never re-fires — and is green with the fix. End-to-end acceptance: the noflo-jekyll stress went from ~10/100 variant builds across three shapes to 45+ consecutive deterministic builds.
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
The one-firing rule's arrival walk fired each ready scope with the arriving IP as the activation context — for the arriving port's own scope that IP is the head, but for any other port's pending scope the context was a lie. The concrete failure (work document #48 update #13, distilled from the noflo-jekyll verification's null-partition report: a control port's buffered openBracket arrived while another port held unscoped work; the bracket became that port's firing context, and fireProcess's openBracket check engaged auto-ordering for the whole component. The drain's sends then queued into the ordered output machinery although no stream ever arrived on the firing port, and the ordered flush re-sent the activation's accumulated result on every deactivation — three logical sends emitted nine packets with a prefix-growing pattern. The walk now offers readyScope.head as the context — the entry the scan already captured for the drain pump — in both the unscoped and the scope-stamped forms. The scan derives the scope from the head IP rather than the buffer's object key: buffer keys are strings, while the scope primitive (integer scopes are contract) must be what the process and its outputs see. The pump's offer path heals identically. The deterministic pin drives the exact testbed shape: unscoped documents held head-of-line on a layout chain whose completing template arrives last in a framed control-stream corpus. It asserts the exact emission sequence — the strand and the duplication are both regressions it catches — and the closedown landing after the drain. Work document #48 update #13. EOF )
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
…d-error detail Review findings on #1103 (work document #48 update #15), three nits resolving together: The component forwarded only inport stranded reports to its own stranded surface; outport-side reports (unattached sends, dropped errors) reached the component surface only through direct dispatches on nodeInstance — which is wired only inside a Network. A standalone component's unattached outport fired the port-level event and nothing on the surface its consumers actually subscribe to. The outport forwarding listener now matches the inport path, and the direct node-level dispatches are gone — the component surface is the single path, so no double-dispatch in networks either. The dropped-error detail carried the raw Error object; the network's emit path spreads the detail into the event payload where every other stranded entry is counts and strings. The error is serialized at the source (error: String(e)), so the payload survives the spread and the tracefile unchanged. The unattached-send port-level payload now carries the port name itself, so the report is self-describing before any forwarding.
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
A component with no error port declared that calls done(err) escalates through a load-bearing chain: error() throws, fireProcess's catch answers with its own sendDone(e) which throws again from inside the catch, the synchronous listener-throw propagation rejects the delivering edge's write, and the socket's error side-channel turns it into the network's process-error — or the loud async throw when no socket error listener exists (standalone). Verified empirically and held by a network-level pin: the chain is exactly what the no-silent-loss invariant needs for components without an error leg, and a refactor of the catch could sever it silently. Work document #48 update #16.
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
The no-silent-loss surface: strands, clobbers, and dropped errors are loud (work document #48)
bergie
added a commit
that referenced
this pull request
Oct 19, 2026
…s errors (work document #35 update #23 review) Two documented-invariant violations the #1106 review verified by live probes, both in CollectFields' merge loop: - The named fields and the spread merged together in connection order, so a same-key collision resolved by JS object key order — ['meta','*'] let the spread win while ['*','meta'] let the named field win, contradicting the documented order-independence. The merge is now three passes — replace, named fields, spread — fixed by mapping kind, never by connection order. - The spread's errors key interacted inconsistently with the error accumulation: silently discarded in the spread-only shape (the final overwrite), clobbering the whole accumulation in the replace+spread shape — a silent loss of diagnostics, the class the #48 no-silent-loss surface exists to surface. The spread now never owns errors: the errors key is excluded from the merge and the accumulation channels own the field, making both config shapes behave identically. The doc comment matches. Plus the review's judgment call: an in-mapping value of '*' is rejected at wrap time — the spread marker is out-mapping vocabulary, and the in-side '*' would generate an extractor walking a literal '*' key and failing at runtime. Pinned: the connection-order independence (both config orders, same result), the errors exclusion (both shapes, plus the error-port append on the replace path under the terminal-event semantics), and the wrap-time rejection. Suites: assembly-utils 47, assembly 49.
AsherBond
pushed a commit
to AsherBond/noflo
that referenced
this pull request
Oct 20, 2026
… loud (work document noflo#48) Implements the invariant the three-shape verification re-opened: the Network must not lose messages silently under any scheduling. Three surfaces, all engine-level and wire-safe (the 0x0f STRANDED tracefile encoding stays bracket-shaped; the data-shaped entries ride the JS event bus pending the freeze discussion): - The closedown residue census: when the closedown predicate fires, the component's undeliverable data residue (per port, per scope, via the new InPort.strandedData()) is emitted as a component 'stranded' event and logged with named counts — the moment the strand became final, not a debug dump at shutdown. Standing IIP buffers are excluded: an unconsumed initial is configuration re-delivered per run (noflo#46), not stranded work. - The control-stream replacement clobber: when a control port's latest-stream rule discards a buffered stream holding packets, the InPort dispatches a stranded report naming the discarded counts — the bang-vs-corpus hazard from the noflo-jekyll verification becomes diagnosable at the replacement moment instead of as a downstream strand. - The unattached-outport send: a send into an outport with no attached sockets vanishes by contract, but the vanish now dispatches a stranded report and logs — the noflo#35 update noflo#18 footgun becomes visible. The component-side answer for legs that must not drop is required: true (escalates in checkRequired). The Network forwards component stranded events onto its event bus; bracket-shaped entries record to Flowtrace under the existing 0x0f encoding, data-shaped entries stay on the JS surface. Three new deterministic pins in spec/wd48-anomaly.js: the closedown census, the replacement clobber, and the unattached-fail-leg drop.
AsherBond
pushed a commit
to AsherBond/noflo
that referenced
this pull request
Oct 20, 2026
In ordered mode every activation pushes its context's result object into outputQ (Component.activate), and a process function that completes its activation per item — a sendDone-in-a-loop drain — re-activates the same context on its continuation: the loop's next read, or simply the closing done(), both call activate on the completed context. The re-activation re-pushed the same result object whose port arrays still held every already-sent IP, so each ordered flush re-sent the accumulated history: three logical sends emitted six-to-nine packets in a prefix-growing pattern (d1), (d1,d2), (d1,d2,d3) — the duplication the noflo-jekyll verification probes caught as d1x4/d2x3/d3x2 during the noflo#48 investigation. noflo#1103 fixed that investigation's spurious entry path into ordered mode (a cross-port context IP engaging auto-ordering for a component no stream ever arrived on); this closes the genuine ordered-mode defect on master. processOutputQueue now empties a result's port arrays as it sends them — the queue holds unsent work only, by construction. Keys are kept (the bracket-forwarding machinery walks them); only the sent entries leave. A re-activated context re-pushing its result is now harmless regardless of how many times it re-activates: the re-push enqueues an empty result. The deterministic pin (added red, green at merge) drives a genuinely auto-ordered component — a real framed stream engaging autoOrdering on its own inport — whose drain loops sendDone per item, and asserts the exact emission sequence: each item once, in order. On master it failed with ["a","a","b","b","c","c"]. Work document noflo#49.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wrap groups on the outside of the incoming connection