8000 Compile subgraphs into parent graphs before executing by kenhkan · Pull Request #42 · noflo/noflo · GitHub
[go: up one dir, main page]

Skip to content

Compile subgraphs into parent graphs before executing - #42

Merged
bergie merged 2 commits into
noflo:masterfrom
kenhkan:precompile_subgraphs
Oct 1, 2012
Merged

bergie merged 2 commits into
noflo:masterfrom
kenhkan:precompile_subgraphs

Conversation

@kenhkan
@kenhkan kenhkan commented Aug 27, 2012
Copy link
Copy Markdown
Contributor

My other two outstanding pull requests (#40, #41) deal with subgraphs. They have always felt like unsafe patches to me, but I think I finally got a cleaner solution to it, but it does change the API so I really look forward to some input.

I figured that the problems that I've been encountering is due to the fact that subgraphs are initialized after the parent graph is set up and ready to send IPs. This solution involves pre-compiling the sub-graphs directly into the parent graph (and do so recursively) before executing the graph.

So, in this completely trivial example:

# abc.fbp
'group' -> GROUP Group(Group)
Merge(Merge) OUT -> IN Group() OUT -> IN Split(Split)
# def.fbp
'abc.fbp' -> GRAPH A(Graph)
Merge(Merge) OUT -> IN A.Merge()
# main.fbp
'def.fbp' -> GRAPH A(Graph)
'initial' -> IN Merge(Merge) OUT -> IN A.Merge()
A.A.Split() OUT -> IN Output(Output)

This would yield (invisible to the programmer):

'group' -> GROUP A.A.Group(Group)
A.A.Merge(Merge) OUT -> IN A.A.Group() OUT -> IN A.A.Split(Split)

A.Merge(Merge) OUT -> IN A.A.Merge()

'initial' -> IN Merge(Merge) OUT -> IN A.Merge()
A.A.Split() OUT -> IN Output(Output)

What this does is eliminate all the testing and waiting for a ready-state. The namespace should also avoid any name clash since FBP doesn't have dots in component names.

The downside of this approach is the change at the API level.

Would this be a long-term solution to the problem?

@travisbot
Copy link
Copy Markdown

This pull request passes (merged 12c1ec3 into 5f4fcfa).

@travisbot
Copy link
Copy Markdown

This pull request passes (merged 707a274 into 5f4fcfa).

@kenhkan
kenhkan commented Sep 1, 2012
Copy link
Copy Markdown
Contributor Author

I have been reading Morrison's book and realized that the concept of a graph is to look like a regular component. Having the dot-notation breaks the black-box-ness of FBP.

So I guess we should always stick with the PROCESS.PORT convention, but I'd still be for the idea of compiling the graphs before execution.

@travisbot
Copy link
Copy Markdown

This pull request passes (merged 8cbfff2 into 5f4fcfa).

bergie added a commit that referenced this pull request Oct 1, 2012
Compile subgraphs into parent graphs before executing
@bergie
bergie merged commit 7f015cf into noflo:master Oct 1, 2012
bergie added a commit that referenced this pull request Oct 12, 2026
…ibution, and the live DATA envelope

Work document #4 update #42, completing update #33's one-identifier-per-
graph-instance promise for subgraphs:

- Subgraph planes are registered at network build: the execution protocol
  walks the built network and registers every subgraph instance anchored
  to its node id (kind subgraph, parent chain, component name), with
  deterministic sg/<parent>/<node_id> ids reconciled per build. The sg/
  prefix is runtime-reserved — client-minted ephemeral ids using it are
  rejected with 0x17 on sync and ops. 0x16 now lists the full plane tree;
  dropping a subgraph plane is refused with 0x17, and dropping an
  ephemeral plane cascades to its subgraph descendants
- Telemetry attributes events to their effective plane: the engine's
  internal subgraph path resolves through the plane registry, so events
  sourced inside a nested subgraph are attributed to the subgraph's plane
  (the path never travels the wire). Per-plane flush grouping emits
  correctly attributed chunks
- Live 0x01 DATA telemetry carries the §7 envelope [src, tgt, value]
  built from the event's socket refs (dataEventEnvelope), with the value
  frugalized before enveloping so unserializable packets cannot take the
  refs down; group events keep their plain label payloads
- The tracefile projection follows the §7 payload rules exactly: the
  envelope is DATA-only — group events keep plain strings, connection
  events project to the flat four-string ref pair
- The bracket/stream event-type vocabulary renames to BEGIN_BRACKET/
  END_BRACKET (codes unchanged; the flowtrace-lineage begingroup/
  endgroup naming stays only in the recorder JSON input format),
  disambiguating from the graph-model group entity (INSERT_GROUP)

SPEC §5/§7 and the AsyncAPI channel descriptions follow; tests: subgraph
plane registration/reconciliation, reserved-prefix rejections, 0x15
refusal and cascade, effective-plane attribution, envelope assertions on
live and real-engine paths, and a full-stack subgraph e2e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

0