8000 Typed Erlang FFI (`extern module` declarations) — "Erlang typing info" · Issue #83 · kexhq/kex · GitHub
[go: up one dir, main page]

Skip to content

Typed Erlang FFI (extern module declarations) — "Erlang typing info" #83

Open

Description

@akoskovacs

Summary

Typed bindings for the Erlang FFI: extern module declarations, a BEAM-shape type vocabulary, arity and argument checking at foreign call sites, using for foreign paths, declaration generation from OTP specs, and Hex packages as a dependency kind.

Current state

The FFI itself works — Erlang.Lists.reverse([1,2,3]) compiles to call 'lists':'reverse' and runs. But:

  1. Every foreign call is untyped. src/semantic/typechecker.cxx:3149-3162 short-circuits any call rooted at Erlang/Elixir/Gleam and returns Type::unknown(). Nothing checks arity, argument types, or the result — an arity mistake is undef at runtime with no source location.
  2. using does not reach foreign modules. using Erlang.Lists warns module not found in source roots, and the prefix must be repeated at every call site.
  3. The walker cannot do it at all. Erlang.Lists.reverse under the default backend reports Undefined function: Erlang.Lists, which reads like a typo.

Core design decisions

  • Declarations describe BEAM shapes, never Kex shapes. A foreign signature says Charlist, not String — Kex String is a binary, while Erlang.Erlang.system_info(:otp_release) returns [50, 57]. Adaptation is ordinary Kex code in a wrapper. A declaration claiming system_info : Atom -> String would be a lie that surfaces as [50, 57] in output or badarg several frames deeper — not as a type error.
  • No new file format. Declarations are ordinary .kex source; a declarations-only package is an ordinary package with no runtime code.
  • Opt-in and additive. An undeclared foreign module keeps working exactly as today, at Unknown.
  • A declaration is trusted, and can be wrong — unsafe in the Rust sense. The mitigation is a debug-mode contract check, not a proof.
  • Foreign calls stay BEAM-only. The walker gets a diagnostic, not an implementation.
  • No implicit build scripts. Compiling plain .erl with erlc ourselves is not an install script; running mix is, and needs to be asked for out loud.

The type vocabulary

Name BEAM value Notes
Binary binary what Kex String already is; the only free conversion
Charlist list of codepoints needs list_to_binary
ErlAtom atom distinct from Kex Atom's naming rules
ErlList<T> proper list element types may need mapping
ErlTuple<A, B, ...> tuple fixed arity
ErlMap<K, V> map
Pid, Ref, Port, Fun opaque no operations except passing back
ErlTerm anything the honest Unknown; explicit rather than inferred
ErlResult<T> {ok, T} | {error, term()} the Erlang convention, not Kex's Result
ErlBool true | false atoms coincides with Kex Bool, converts freely

The declaration form

extern module Erlang.Lists do
  reverse : ErlList<ErlTerm> -> ErlList<ErlTerm>
  sum     : ErlList<Integer> -> Integer
  seq     : (Integer, Integer) -> ErlList<Integer>
end

extern module Erlang.GenServer, atom: "gen_server" do
  call       : (Pid, ErlTerm) -> ErlTerm
  start_link : (ErlAtom, ErlTerm, ErlList<ErlTerm>) -> ErlResult<Pid>
end

The atom: override fixes the Erlang.Gen_server wart: underscored module names must be written with exactly one capital (Gen_server, not GenServer — the latter lowercases to genserver, which does not exist), and the failure arrives as undef with no hint. The declaration names the BEAM atom exactly; the Kex-side spelling becomes free.

Typechecker seam

src/semantic/typechecker.cxx:3149-3162 is the whole change surface:

  • found, arity matches — check arguments against the declared parameters, return the declared result type
  • found, arity does not match — error naming the declared arity (the single highest-value diagnostic: an arity mistake on the BEAM is undef at runtime with no source location)
  • not found — today's behaviour, unchanged

--check-extern wraps each declared foreign call in a shape assertion (is_binary, is_list, is_pid, tuple arity) on arguments and result, reporting mismatches against the declaration with its source location. Off by default.

using for foreign paths

  • src/semantic/resolve_pass.cxx:244 must not warn for a path rooted at Erlang/Elixir/Gleam — there is no source root to find it in, and that is not an error
  • The alias must reach modulePath in src/ir/lower.cxx:2287, which already consults moduleAliases for user modules
using Erlang.Gen_server, as: GenServer

GenServer.call(pid, msg)

Walker

Report "a foreign call requires the BEAM backend — use -R or -c" at src/interpreter/evaluator.cxx:1926, which already has the namespace name in hand. A package whose public API bottoms out in extern is BEAM-only, and tey test on it must run under -R.

Declaration generation

tey extern gen <module-or-package>: read exports and specs from the compiled .beam (beam_lib:chunks/2 for Dbgi/Docs, or Code.Typespec.fetch_specs/1 for Elixir), map onto the vocabulary, emit ErlTerm with a # TODO for everything unmapped, write extern/<module>.kex with atom: filled in whenever the lowercasing rule would not reproduce the module name. Generated output is a starting point a human narrows — same relationship @types packages have with the JS they describe.

Hex dependencies

bundle "myapp" do
  tey("greet", git: "https://github.com/kexhq/greet", tag: "~> 0.2")

  hex("cowboy", "~> 2.12")                     # Erlang, built with erlc
  hex("jason",  "~> 1.4", build: "mix")        # Elixir, needs a real build tool
end
  • Tey.Manifest.Dependency gains a source kind; Tey.Manifest.read dispatches on the leading word, so a hex( branch is a small change
  • Hex is a plain HTTP repository (repo.hex.pm tarballs + protobuf version list); Tey.Semver already parses Hex's constraint syntax
  • Tey.Cache gains a parallel content-addressed key; CHECKSUM verified against the registry, hash recorded in tey.lock — same guarantee the git path gets from git archive
  • Plain Erlang packages (src/*.erl + .app.src, no plugins, no parse_transform, no NIFs) compile with erlc directly
  • Anything else requires an explicit build: "rebar3" / build: "mix" — a per-dependency grant to execute the package's build definition; without the flag, a layout erlc cannot handle is a hard error naming what it found
  • kex --beam-path <dir> appends -pa when -R spawns erl; Tey.Commands passes dependency beam paths alongside source roots

Sequencing

  1. Make bodiless signatures authoritative for calls — prerequisite and standalone bug fix (below)
  2. extern module + vocabulary + the typechecker seam — the arity check alone justifies the step
  3. using for foreign paths + the walker diagnostic
  4. tey extern gen — turns 1–2 into a feature that pays for itself against real OTP modules
  5. hex(...), fetch, verify, lock — no building yet
  6. erlc building + --beam-path
  7. build: "rebar3" / build: "mix" — last, because it runs other people's code

Prerequisite (standalone bug)

Bodiless signatures are collected but do not drive checking — this passes -C clean and then fails at runtime with Undefined function: Foo.bar:

module Foo do
  bar : Integer -> Integer
end
main do
  IO.printLine(Foo.bar("nope"))   # not reported
end

Non-goals

Sound typing of foreign calls; declared coercions between BEAM and Kex shapes; making the walker call foreign code; NIFs/port drivers; publishing Kex packages to Hex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0