8000 Remove module-level foulness: `foul` should be per-function only · Issue #130 · kexhq/kex · GitHub
[go: up one dir, main page]

Skip to content

Remove module-level foulness: foul should be per-function only #130

Closed

Description

@akoskovacs

Problem

foul module M do … end marks every member foul in bulk. That has two bad
consequences:

  1. let silently means foul. Inside a foul module, let read(path) = …
    is a foul function even though the source says let. The keyword that
    everywhere else in the language means "pure" means the opposite here,
    purely because of an enclosing declaration that may be hundreds of lines
    away. src/stdlib/fs.kex is the extreme case: 70 let definitions, 2
    foul keywords (foul module File, foul module Directory) — so nearly
    every function in the file is foul with nothing at its own definition site
    saying so.

  2. It is too coarse to be true. Most "foul" modules are mostly pure.
    Kex (src/stdlib/kex.kex:47) is declared foul module yet the majority
    of its exports do no IO — which is why the analyzer already carries a
    hard-coded exemption for it:

    // src/semantic/analyzer.cxx:388
    // Kex.Intrinsic.* is the private ABI — its per-export isFoul
    // governs purity, not the blanket Kex module foulness.
    bool isIntrinsicCall = (node.method == "Intrinsic");

    A special case in the purity checker to undo module foulness is the sign
    that the granularity is wrong. Blanket module foulness also makes
    cross-module calls conservative: isModuleFoul(receiver) rejects a call
    from pure context based on the module's flag, without ever consulting the
    callee's own effect.

  3. foul do … end blocks have exactly the same defect — and worse, they
    are documented but were never implemented. docs/purity.md:32,
    docs/concurrency.md:91 and docs/streams.md:79 all show a bare foul do
    block wrapping let definitions, but the parser has no such form:

    $ kex foul_do.kex
    foul_do.kex:1:6: error: Expected function name
    

    There is no FoulBlock AST node and no parse path — foul at top level
    goes straight to parseFunctionDef(true). So this is documentation for a
    feature that does not exist, and it should not be built: a block that makes
    the lets inside it foul is th 10BC8 e same action-at-a-distance as the module
    form.

Effects should be visible where the function is defined, not inherited from an
enclosing scope. Every function should say what it is at its own site.

Proposal

Remove module-level foulness and do not introduce block-level foulness.
foul becomes a per-function marker, and nothing else.

  • Drop foul before module from the grammar and parser, with a parse error
    pointing at per-function foul. Drop the never-implemented foul do … end
    block from the docs and grammar rather than implementing it.
  • Remove ModuleDef::isFoul and the module-foulness path in the analyzer
    (m_symbols.pushScope(mod.isFoul), the m_inFoulContext set on module
    entry, isModuleFoul in the UFCS call check).
  • Purity of a cross-module call is then decided solely by the callee's own
    isFoul, resolved through the imported interface's per-export flag
    (KexI already records this: kexi.cxx:195/215). Delete the
    Kex.Intrinsic special case — it stops being needed.
  • Retire the module-level foul field in the .kexi interface format and its
    chunk metadata (kexi.cxx:260/280/385/470/675), or keep it written-as-false
    for one release if the format needs a compat window.

Migration

Every foul module in the tree gets rewritten so each genuinely effectful
member is declared foul name(…) instead of let name(…); the pure members
stay let and become correctly callable from pure code.

Sites: src/stdlib/{env,http,fs,io,kex,process,system}.kex,
examples/streams.kex:28.

This is the interesting part of the work — deciding, per function, whether it
actually performs an effect. Expect it to reclassify a good number of stdlib
functions as pure that are foul today only by inheritance.

Docs to update

docs/purity.md (delete both the "Per block" and "Per module" entries from
the granularity section — only "Per function" survives),
docs/concurrency.md:91, docs/streams.md:79 (both use foul do blocks that
never parsed),
docs/modules.md:62 ("Foul Modules"), docs/language-spec.md:1213
("Modules may be marked foul module, making every member foul"),
docs/module-sys-plan.md, docs/plan_tey_pm.md.

Activity

  1. added this to the 0.4.0 milestone on Aug 19, 2026
  2. akoskovacs commented on Aug 19, 2026

    @akoskovacs
    CollaboratorAuthor

    Implemented on remove-module-foulness (working tree, uncommitted). make test-all is green: 19/19 ctest suites, 259 interpreter specs, 458+51 prelude specs, 449+51 BEAM specs, 0 failures.

    Three things the implementation turned up

    1. Qualified-call purity was never actually enforced per function — only per module. There was no path from M.f(...) to f's own effect. Removing module foulness therefore could not just delete code; it needed a new check. Analyzer::isQualifiedCallFoul now answers it, backed by isExportFoul for imported modules and a new collectModuleMemberEffects prepass (Phase 0.5) for modules declared in the same unit — the prepass runs before scope resolution so a call can reference a module declared further down the file.

    2. Module foulness was not even enforced consistently. Only single-segment receivers were checked, so a nested foul module was wide open. On main before this change:

    let bad(p: String) = FS.File.read(p)   # accepted, no purity error
    

    receiverModuleName now walks the whole dotted path, so FS.File.read is rejected from pure code. (It matches the receiver exactly as written and does not try to resolve a shortened path — matching any module whose name ends in a segment would make Mock.System.OS in http.kex poison every System.OS read. Missing a rejection is recoverable; rejecting pure code because an unrelated module shares a segment is not.)

    3. foul cannot go on an annotation, so the interface builder has to carry it over. foul name : T is the foul VALUE-binding form (foul lines : Feed<String>), so an arrow signature cannot carry the marker — foul read : FilePath -> String? typechecks as a value of function type and fails against the definition's arity. Annotated stdlib functions get their exported signature from the annotation, so without a fix every annotated function exported as pure. prelude_interfaces.hxx now ORs the definition's isFoul onto the annotation-derived signature.

    Classification rule used for the migration

    foul = touches the outside world at call time; pure = fixed facts about the build or machine that cannot change during a run.

    Reclassified as pure (foul before only by inheritance): all of FS.Path, System.OS/BITWIDTH/macOS?/linux?/windows?/posix?, Kex.BACKEND/interpreted?/underBeam?/Kernel.VERSION/Feature.*, and IO.inspect (already documented as purity-exempt). The Kex.Intrinsic special case in the analyzer is gone — it was only ever there to undo blanket Kex foulness, and per-export purity makes it unnecessary.

    Newly caught as genuinely foul: OptionParser.run and OptionParser.printHelp do IO and were passing as pure, because the old check consulted imported interfaces that are not yet populated while the prelude itself is being built. The local-module path closes that bootstrap hole.

    Also changed

    • foul module is a parse error naming the fix. The foul token is consumed BEFORE reporting: Parser::error throws and syncToTopLevel treats foul as a resume point, so erroring first spun the recovery loop forever (caught as a hung parser_test eating 8.5 GB).
    • sourceParameterNames in the LSP required let after an optional foul prefix. foul REPLACES let, so every respelled stdlib function lost its completion parameter names. Fixed.
    • ModuleDef::isFoul, ImportedModuleInterface::isFoul and KexiStructuralMetadata::isFoul are gone, along with the foul key in the .kexi structural metadata map. Dropping a map key is compatible in both directions — older files keep the key and it is ignored.
    • tey/'s six modules used standalone foul module. These were migrated mechanically (every member marked foul), which preserves today's semantics exactly but is the one place I did not audit function by function. Likely pure and worth narrowing later: Tey.Resolver.lowestSatisfying/otpReason, Tey.Git.tagName, Tey.Cli.versionLine/unknown, Tey.Commands.moduleNameFor.

    Not done

    make Pid/make Task/make Reference receiver methods (send, link, unlink, monitor, alive?, await, demonitor) are process operations that are pure today — they sit outside any module, so module foulness never covered them. Marking them foul is correct but is a behavior change beyond removing module foulness, and would reject existing code. Separate ticket.

  3. akoskovacs commented on Aug 19, 2026

    @akoskovacs
    CollaboratorAuthor

    tey audited function by function

    The mechanical migration noted above is replaced by a real audit. Rather than reading 2000 lines by eye, I demoted every tey function back to let and let the purity checker report what genuinely needs foul, iterating to fixpoint — the minimal correct foul set by construction.

    This immediately exposed a hole in the checker itself. Analyzer::analyzeModule skipped VisibilityBlock with a // TODO: handle visibility, so a private do member's body was never purity-checked at all. Visibility decides who may CALL a function, not whether its body is checked. With the block skipped, toolchain.kex's private block alone hid 62 purity errors, and the first audit run happily reported download, installArchive, fetchRelease, link, publish and writeEscript as pure. Fixed in this change:

    • analyzeVisibilityBlock now analyzes the block's function/type/record/make items.
    • collectModuleMemberEffects descends into visibility blocks, so a private member's foul marker is registered — otherwise a caller in the same module sees it as pure.

    Also needed: dispatchCommand's eleven overload clauses have to agree on purity ("Overloads of X must all have the same purity"), so per-call-site marking had to be normalized per function name.

    Result — genuinely pure (13 of ~90)

    module pure
    Tey.Git tagName
    Tey.Cache packagePath, stagingPath
    Tey.Resolver chooseTag, effectiveOtpFloor, lowestSatisfying, otpReason, resolveDependency
    Tey.Toolchain newestFirst, versionOf
    Tey.Cli parser, versionLine
    Tey.Commands moduleNameFor, outputDirectory, readme

    Everything else is foul, transitively verified. Of the six I guessed at earlier: lowestSatisfying, otpReason, tagName, versionLine and moduleNameFor are indeed pure; Tey.Cli.unknown is not — it prints to stderr.

    Note Tey.Cache.sourceRoot is foul (it reads Tey.Cache.root, which reads ENV) while packagePath and stagingPath, which call it, are pure — they only build strings from an already-resolved path.

    Verification

    kex -C --source-root src reports zero purity errors across all of tey/. Thirteen non-purity errors remain (ManifestPackage field access in resolver.kex, targets/entrypoint in commands.kex); these are pre-existing and unrelated — checking HEAD with only the foul module keyword dropped and no other edit produces the identical thirteen.

    make test-all still green after the analyzer change: 19/19 ctest, 259 interpreter specs, 458+51 prelude, 449+51 BEAM, 0 failures.

  4. akoskovacs commented on Aug 19, 2026

    @akoskovacs
    CollaboratorAuthor

    Correction to the tey audit above

    CI caught a real error in my audit. The fixpoint I ran used per-file kex -C --source-root src, and that does not resolve sibling modules — so every cross-module call inside tey went unchecked. Three functions in Tey.Resolver were wrongly classified pure:

    function foul because
    effectiveOtpFloor reads Tey.Toolchain.runtimeOtpFloor
    chooseTag calls Tey.Git.tags
    resolveDependency calls Tey.Git.pinRef, Tey.Cache.fetch, Tey.Git.execute

    The whole-program compile (kex --compile src/main.kex, which is what make -C tey build runs) does resolve them and reported all five call sites. Re-running the fixpoint against that instead reaches a fixpoint in one pass with exactly these three changes.

    So the correct pure set is ten, not thirteen — drop chooseTag, effectiveOtpFloor and resolveDependency from the table above. lowestSatisfying and otpReason stay pure.

    Verified: make -C tey build succeeds, make spec-tey is 87 passed / 0 failed, and the tey CLI smoke test passes — the three steps the tey CI job runs, the first of which was the failure.

    Method note for anyone doing this again: audit purity against the whole-program build, not per-file -C. Per-file checking silently skips cross-module purity, and because it reports fewer errors it looks like success.

  5. akoskovacs commented on Aug 19, 2026

    @akoskovacs
    CollaboratorAuthor

    Filed #142 for the one purity hole this change does not close: Block<Void> carries no effect in its type, so a foul block manufactured in a foul context can be run by a pure higher-order function. Pre-existing — module foulness never addressed it either — and load-bearing, since Enumerable.each being pure while accepting foul blocks is what makes xs.each do IO.printLine(x) end work at all.

  6. akoskovacs commented on Aug 19, 2026

    @akoskovacs
    CollaboratorAuthor

    Also filed #143 — inferred capabilities plus a lexically scoped with override, as a successor to this work. It replaces the global Mock.* state (which forces before/after clearing and blocks parallel specs) with a scope that dies with its block. Depends on widening isFoul from a bool to a set of atoms, which reuses the fixpoint in computeTransitiveEffects and delivers the target capability declarations docs/purity.md already promises. Independent of #142.

  7. added a commit that references this issue on Aug 19, 2026
    be24a53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0