8000 `kex --emit-ast` and `kex_intrinsic_ast.erl`: AST access on BEAM · Issue #134 · kexhq/kex · GitHub
[go: up one dir, main page]

Skip to content
  • kex --emit-ast and kex_intrinsic_ast.erl: AST access on BEAM #134

    Closed

    Description

    @akoskovacs

    Part of #131. Blocked by #133.

    Parser exists only in the tree-walk interpreter. Under -R it raises undef — there is no runtime module behind it — and Tey, the first consumer that needs it, runs on BEAM. This is the third and largest of the blockers recorded in docs/tey-limitations.md.

    The approach is a call out to the compiler: kex parses, emits the AST as a term, and the Kex side pattern-matches it. No NIF, no second FFI, no C++ crash taking the VM down.

    What lands

    kex --emit-ast <file> writes the ETF encoding of the AST to stdout (-o for a file), using TermBuilder from #133.

    The term is encoded directly in Kex's own BEAM value representation — variants as {'Call', ...}, records tagged with their qualified name ({'Kex.AST.Program', ...}), String as a binary, Char as {'Char', N}, Optional as 'None' / {'Just', X}. So the Erlang side is binary_to_term and nothing else, and Kex code matches the result as ordinary values. The alternative — a neutral term tree plus a Kex-side decoder — costs a decoder that must be kept in step with the schema, for no gain.

    runtime/src/kex_intrinsic_ast.erl locates the kex executable, runs it through a port the way kex_intrinsic_process:run/2 already does, and answers {'Ok', Term} or {'Error', ParseError}. A non-zero exit or unparseable output becomes a ParseError. runtime/src/kex_intrinsic_interface.erl is the model to follow, including its note on why binary_to_term's safe option is unusable here: the payload is mostly atoms the running VM has never seen, which is precisely what it is for.

    Deployment constraint

    This makes kex a runtime dependency of any BEAM program that parses Kex, not just a build-time one. Resolution order: $KEX, then os:find_executable("kex"), then a clear error naming both. This is fine for Tey — Tey owns and installs every Kex toolchain — but it is a real constraint and is recorded here rather than discovered later.

    Verification

    • make test-all, plus every existing parser spec re-run under -R. From this point on that dual run is the drift alarm for all later stages.
    • kex --emit-ast examples/records.kex | erl -noshell -eval 'io:format("~p~n",[binary_to_term(...)])' — spot-check the shape by hand once.
    • A spec asserting the interpreter and -R produce the same AST for the same source.

    Activity

    1. added a commit that references this issue on Aug 21, 2026
    2. added this to the 0.4.0 milestone on Aug 21, 2026
    3. akoskovacs commented on Aug 21, 2026

      @akoskovacs
      CollaboratorAuthor

      Done in e5338ad (which said fixes #134 but never triggered the close). kex --emit-ast is in the CLI, and runtime/src/kex_intrinsic_ast.erl ships as runtime/beam/kex_intrinsic_ast.beam, so Kex.AST.parse works under -R — spec/stdlib/ast.spec.kex passes on both backends.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Type

      No type

      Projects

      No projects

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions

      • 0