8000 Bracket file data in group named with filename. by rybesh · Pull Request #16 · noflo/noflo · GitHub
[go: up one dir, main page]

Skip to content

Bracket file data in group named with filename. - #16

Merged
bergie merged 2 commits into
noflo:masterfrom
rybesh:readfile
Apr 3, 2012
Merged

bergie merged 2 commits into
noflo:masterfrom
rybesh:readfile

Conversation

@rybesh
@rybesh rybesh commented Mar 30, 2012
Copy link
Copy Markdown
Contributor

If you're sending a bunch of filenames to ReadFile, it seems useful to be able to keep track of which files the data came from. If I'm overlooking a better way of handling this, please let me know.

bergie added a commit that referenced this pull request Apr 3, 2012
Bracket file data in group named with filename.
@bergie
bergie merged commit 0555ddc into noflo:master Apr 3, 2012
bergie added a commit that referenced this pull request Oct 9, 2026
Phase 2 of work document #28: RegistryProtocol, the Two-Step Cache server
side, driven end-to-end through the runtime core in tests.

The runtime consumes an application-supplied catalog — the same philosophy
as the engine's application-supplied component registry (work documents
#6/#16): signatures are ready at handoff, and the handler derives the wire
state from them. A signature is exactly the 0x24 detail shape (declared
kind plus the full port field set), valid with no implementation behind
it, so manifest/sidecar-sourced stubs flow through unmodified.

- 0x20 syncs answer 0x21 on hash match, 0x22 manifest otherwise — the
  runtime always answers (work document #4 update #18)
- 0x23 details answer with only the requested known components; unknown
  names are omitted, and an empty response is an answer
- 0x25 source writes and 0x27 installs map onto optional catalog hooks
  (source and signature are orthogonal, so a write never relabels a
  component); missing hooks surface as unsupported events
- every catalog change re-derives sig_hash values and the registry_hash
  through the protocol package's canonical serializations

Supporting changes: the runtime core now accepts async handlers (WebCrypto
hashing is async), capturing rejections as error events like synchronous
throws; @noflo/fbp-protocol re-exports its ComponentDetail/PortInfo/
ManifestEntry typedefs from the entrypoint so dependents can type against
them.
bergie added a commit that referenced this pull request Oct 9, 2026
Phase 4 of work document #28: the 0x30/0x32 streaming telemetry surface,
fed by a real NoFlo network.

NetworkHost owns the network lifecycle: builds the NoFlo network from the
runtime's graph with an injected component loader (platform-neutral, per
work documents #6/#16), re-emits the network's EventTarget events so
protocol handlers observe a network before, after, and across network
recreations, and supports rebuild-on-discard. The engine gap it surfaces
is documented: the 2.x engine consumes the graph at connect() and does
not follow live graph changes, so CRDT operations applied while a network
runs update the wire state but not the running network until the engine
learns to follow graph deltas.

TelemetryProtocol maps host events onto the 0x32 vocabulary:
- data packets become DATA events with raw payloads; brackets become
  BEGIN_GROUP/END_GROUP with the group name
- network start/stop become LIFECYCLE START/STOP codes
- process errors become ERROR events with the exception string — except
  errors in the stub error class (default StubNotImplementedError), which
  become STUB_ERROR: the event type itself is the classification (#4
  update #9)
- payloads the wire cannot carry are frugalized to type-named
  placeholders so serialization can never take down the flush loop

Flush policy is the runtime's physical decision: per-subscription buffers
flush on timers at the effective cadence (the client's requested
interval, bounded below by a configurable floor), with explicit flushing
available for transports that prefer it. The protocol has no unsubscribe
command; subscriptions live with their link, and dropContext tears them
down on link close.

The integration test streams a real network run — a real ComponentLoader,
asComponent factory, graph with IIP — through the full pipeline to codec
round-trips. @noflo/noflo re-exports the Network class, restoring 1.x
surface parity that the host needed to type against.
bergie added a commit that referenced this pull request Oct 9, 2026
Phase 2 of work document #28: RegistryProtocol, the Two-Step Cache server
side, driven end-to-end through the runtime core i
8000
n tests.

The runtime consumes an application-supplied catalog — the same philosophy
as the engine's application-supplied component registry (work documents
#6/#16): signatures are ready at handoff, and the handler derives the wire
state from them. A signature is exactly the 0x24 detail shape (declared
kind plus the full port field set), valid with no implementation behind
it, so manifest/sidecar-sourced stubs flow through unmodified.

- 0x20 syncs answer 0x21 on hash match, 0x22 manifest otherwise — the
  runtime always answers (work document #4 update #18)
- 0x23 details answer with only the requested known components; unknown
  names are omitted, and an empty response is an answer
- 0x25 source writes and 0x27 installs map onto optional catalog hooks
  (source and signature are orthogonal, so a write never relabels a
  component); missing hooks surface as unsupported events
- every catalog change re-derives sig_hash values and the registry_hash
  through the protocol package's canonical serializations

Supporting changes: the runtime core now accepts async handlers (WebCrypto
hashing is async), capturing rejections as error events like synchronous
throws; @noflo/fbp-protocol re-exports its ComponentDetail/PortInfo/
ManifestEntry typedefs from the entrypoint so dependents can type against
them.
bergie added a commit that referenced this pull request Oct 9, 2026
Phase 4 of work document #28: the 0x30/0x32 streaming telemetry surface,
fed by a real NoFlo network.

NetworkHost owns the network lifecycle: builds the NoFlo network from the
runtime's graph with an injected component loader (platform-neutral, per
work documents #6/#16), re-emits the network's EventTarget events so
protocol handlers observe a network before, after, and across network
recreations, and supports rebuild-on-discard. The engine gap it surfaces
is documented: the 2.x engine consumes the graph at connect() and does
not follow live graph changes, so CRDT operations applied while a network
runs update the wire state but not the running network until the engine
learns to follow graph deltas.

TelemetryProtocol maps host events onto the 0x32 vocabulary:
- data packets become DATA events with raw payloads; brackets become
  BEGIN_GROUP/END_GROUP with the group name
- network start/stop become LIFECYCLE START/STOP codes
- process errors become ERROR events with the exception string — except
  errors in the stub error class (default StubNotImplementedError), which
  become STUB_ERROR: the event type itself is the classification (#4
  update #9)
- payloads the wire cannot carry are frugalized to type-named
  placeholders so serialization can never take down the flush loop

Flush policy is the runtime's physical decision: per-subscription buffers
flush on timers at the effective cadence (the client's requested
interval, bounded below by a configurable floor), with explicit flushing
available for transports that prefer it. The protocol has no unsubscribe
command; subscriptions live with their link, and dropContext tears them
down on link close.

The integration test streams a real network run — a real ComponentLoader,
asComponent factory, graph with IIP — through the full pipeline to codec
round-trips. @noflo/noflo re-exports the Network class, restoring 1.x
surface parity that the host needed to type against.
bergie added a commit that referenced this pull request Oct 10, 2026
Networks no longer fall back to graphMetadata().componentLoader when no
loader is passed via options; the deprecated path warned since the #16
registry work. Loaders come from network options or the application
registry.
bergie added a commit that referenced this pull request Oct 10, 2026
Since the registry-only loader (#16) the catalog is populated
synchronously at construction and listComponents() is a plain
Promise.resolve -- awaiting it before connect() is a vestige of the
fbp-manifest discovery era.

- NoFlo.createNetwork and Subgraph.createNetwork no longer call it;
  the subgraph keeps a microtask hop so the 'network' lifecycle event
  stays observable to listeners attached right after construction
- Specs: drop the warm-up calls from before() hooks (the ones asserting
  the catalog API itself keep them)
AsherBond pushed a commit to AsherBond/noflo that referenced this pull request Oct 12, 2026
AsherBond pushed a commit to AsherBond/noflo that referenced this pull request Oct 12, 2026
Work document noflo#16, step 1 of the registry-only core split. The
function-wrapping helper was the only user of the get-function-params
dependency, which now leaves noflo core entirely.

- New workspace package @noflo/as-component (EUPL-1.2) holding
  asComponent and its spec; takes noflo as a peer dependency and is
  never imported back by core
- noflo drops the noflo.asComponent export and the
  get-function-params dependency
AsherBond pushed a commit to AsherBond/noflo that referenced this pull request Oct 12, 2026
Work document noflo#16, step 2 of the registry-only core split. The Node.js
discovery machinery (fbp-manifest listing, fbp.json cache, on-the-fly
TypeScript transpilation, CommonJS evaluation, custom noflo.loader
plugins, source storage) becomes a standalone ComponentRegistry
implementation.

- createNodeModulesRegistry(baseDir, options) discovers components and
  resolves to a ready registry: list() is synchronous, get(name) stays
  async per the registry contract
- Discovered graph files register as pre-parsed GraphModel instances
- setSource stores, transpiles, and evaluates source, dispatching a
  change event; getSource includes fbp-spec tests where discovered
- noflo-cache-preheat bin moves here; loadGraphFile/saveGraphFile move
  here from core (EUPL-1.2, greenfield package)
AsherBond pushed a commit to AsherBond/noflo that referenced this pull request Oct 12, 2026
The component class wrapping child networks was Graph in a file named
Graph.js, which collides conceptually with the @noflo/graph GraphModel.
Renames the class and file to Subgraph; the loader catalog entry that
classic discovery registers follows suit.

Work document noflo#16 prepares this by making the loader instantiate the
wrapper directly; this rename is the mechanical part, landing first so
the registry-only core change stays a pure semantic step.
AsherBond pushed a commit to AsherBond/noflo that referenced this pull request Oct 12, 2026
Work document noflo#16, final step. Deletes the whole NodeJs-loader concept
from core: no path strings, no source evaluation, no CommonJS, no
transpilation, no discovery.

Core consumes an application-supplied ComponentRegistry plus manual
registerComponent calls:

- ComponentLoader constructor is options/registry-only; the registry
  catalog is read synchronously at construction (registries are ready
  at handoff), listComponents stays as the noflo#8 promise accessor
- registry EventTarget change/invalidate events keep the loader cache
  current for live systems (FBP runtimes, IDE sessions)
- setSource/getSource/getLanguages/dynamicLoad and the
  sourcesForComponents/specsForComponents scratch fields leave the
  loader; source storage is registry API only
- baseDir exits the loader, networks, and asCallback: pass
  componentLoader or registry instead; subgraph child networks inherit
  the parent loader
- loadGraph instantiates the Subgraph wrapper directly; the Graph
  catalog entry is gone and the runtime graph inport on
  directly-instantiated subgraphs warns deprecated
- isGraph recognizes live GraphModels, FBP JSON, and legacy
  processes/connections JSON shapes
- fbp-manifest, @noflo/fbp, and the noflo-cache-preheat bin leave the
  package; core dependencies are @noflo/graph and debug
- spec fixtures convert from .fbp DSL to committed FBP JSON so the
  parser leaves the devDependency list too
- fbp-spec-runner compiles suites through @noflo/loader-node discovery
  of the project under test
bergie added a commit that referenced this pull request Oct 19, 2026
A component with no error port declared that calls done(err) escalates
through a load-bearing chain: error() throws, fireProcess's catch
answers with its own sendDone(e) which throws again from inside the
catch, the synchronous listener-throw propagation rejects the
delivering edge's write, and the socket's error side-channel turns it
into the network's process-error — or the loud async throw when no
socket error listener exists (standalone). Verified empirically and
held by a network-level pin: the chain is exactly what the
no-silent-loss invariant needs for components without an error leg,
and a refactor of the catch could sever it silently.

Work document #48 update #16.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0