FFFF GitHub - ynishi/htl: Holistic Typed Lua: Teal on mlua, hidden behind cargo · GitHub
[go: up one dir, main page]
Skip to content
ynishi
htl
Public

About

Holistic Typed Lua: Teal on mlua, hidden behind cargo

Resources

Readme

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

2 forks
Report repository
BranchesTags

Repository files navigation

htl — Holistic Typed Lua

Teal (typed Lua) with the toolchain hidden behind cargo. One binary type-checks, lints, formats, tests, bundles and runs .tl; proc macros make Teal type errors fail cargo build and turn Teal's records and functions into Rust; one resolver puts .tl modules into mlua-pkg's require chain. The Teal compiler (tl.lua) is embedded in the mlua state — there is no luarocks, no tl CLI, no generated .lua in your tree.

scripts/foo.tl ──include_tl!──▶ cargo build   (Teal type error = rustc error, with span)
               ──htl run ─────▶ check → gen → load, in one mlua state
               ──htl build────▶ stripped Lua 5.4 bytecode bundle (.hb), no source shipped
Rust impl Host ──#[host_module]▶ UserData impl + host.d.tl   (Rust signature change breaks .tl at build)
               ──#[c_export]──▶ extern "C" wrappers + host.h   (feature `ffi`: a caller that is not Rust)
src/app.tl     ──teal_types!──▶ Rust types + a typed async client on htl::run
                                (Teal signature change breaks the Rust call at build)

The boundary is declared on one side and checked on the other, in both directions: #[host_module] when Rust owns it (functions Teal calls), [rust] exports when Teal owns it (functions Rust calls) — so a host can keep its logic in Teal and its Rust to main and the infrastructure (Calling Teal from async Rust, below).

Install

cargo install htl-cli          # binaries: htl, cargo-htl  (so `cargo htl <verb>` works)

Or a prebuilt binary from the latest release — macOS (Apple Silicon, Intel), Linux (aarch64, x86_64) and Windows (x86_64). The shell and PowerShell installers put htl and cargo-htl in $CARGO_HOME/bin, where cargo install would; Homebrew puts them in its own prefix:

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/ynishi/htl/releases/latest/download/htl-cli-installer.sh | sh
brew install ynishi/tap/htl-cli
powershell -ExecutionPolicy Bypass -c "irm https://github.com/ynishi/htl/releases/latest/download/htl-cli-installer.ps1 | iex"

Those put one htl on a machine. Two projects asking for two — which [toolchain] htl (see htl.toml) refuses by design, naming cargo install htl-cli --version as the way out, a swap for the second project — hand the per-directory part to a version manager. mise needs nothing from htl: its cargo: backend installs from crates.io (the prebuilt binary when cargo-binstall is present), github: takes the release asset, and either writes the pin into the project's mise.toml and switches on cd:

mise use cargo:htl-cli@0.19.0         # mise.toml: "cargo:htl-cli" = "0.19.0"
mise use github:ynishi/htl@v0.19.0    # the prebuilt binary from the release

htl new writes that mise.toml (the cargo: form, at the version that wrote the project) when the CLI is a published release, so a scaffolded project carries its pin from the start; the file is inert without mise, delete it if that is not in use.

[dependencies]
htl = "0.19"                   # embedding: engine + proc macros in one import
crate role
htl umbrella: re-exports htl-core and (feature macros, default on) the proc macros. Depend on this one.
htl-core engine: Htl, lints, fmt, bundle, test runner, mlua-pkg resolver, the generators behind htl rs / htl shapes, and (feature async) the runner htl::run
htl-macros include_tl! / include_tl_bytes! / include_bundle! / teal_types! / TealRecord / host_module / c_export; generated code targets ::htl::
htl-cli the htl / cargo-htl binaries
htl-std std.fs / std.proc, the asynchronous slice of std.*
htl-mq a window for a Teal program (htl new --target window)

htl rs, teal_types!, the [rust] table of htl.toml (modules, exports, [rust.types]) and htl::run are in 0.19.0 and later; on 0.18, htl rs is an unknown subcommand and an htl.toml with a [rust] table does not parse.

CLI

Each command's arguments, flags and examples are in htl <cmd> --help (and htl <cmd> <subcmd> --help for pkg, types, cache and bundle).

command what it does
htl new create a new project directory (see Layout of a project)
htl init fill in missing scaffold files in an existing directory (see Layout of a project)
htl pin move the htl a scaffolded project depends on between release, main and path:<checkout> (see Running against an unpublished htl)
htl check type-check .tl files and directories, replaying unchanged modules from .htl/ (see Lints, Caching)
htl run check then execute a .tl script or .hb bundle on the executor (see Async)
htl test run every .tl that loads the test library in an isolated state per file (see Tests, Caching)
htl fix apply the mechanical fixes diagnostics carry (see Fixing)
htl fmt format .tl files in place (indentation from the syntax tree, blank lines, trailing whitespace)
htl gen emit readable Lua for one .tl file, the escape hatch out of htl
htl build link an entry's require closure into one stripped-bytecode .hb bundle (see Bundles, Caching)
htl bundle inspect what a .hb bundle records (htl bundle info) without running it (see Bundles)
htl unused report modules and [deps] no entry's require closure reaches (see Unused)
htl adopt report how many declarations carry each of htl's nine markers, and where (see Adoption)
htl resolve show which file require("<module>") resolves to, and what it shadows (see Which file a name resolves to)
htl pkg manage mlua-pkg.toml dependencies into .htl/modules/ (install, add, update, clean, patch; see Patched dependencies)
htl types bring declarations for a library from teal-types into types/ (htl types add)
htl cache report what the .htl/cache store holds, or empty it (status, clear; see Caching)
htl dts write the .d.tl files this project declares from Rust (#[host_module], #[derive(TealRecord)]), Teal (---@contract) and crate dependencies (htl::dep_dts)
htl shapes write an lshape run-time schema and typed Schema<T> .d.tl for each record and enum the project declares (htl::shapes)
htl rs write one Rust file: a TealRecord type for each record and enum of the modules in [rust] modules, a client for each module in [rust] exports (htl::rsgen)
htl lsp run the language server on stdio, whose diagnostics are htl check's (see Editors)

Exit code 1 is a verdict (an error in a file, a finding at deny, a failing test, a name that resolves to nothing); 2 is a command that could not reach one (a directory in no project, an htl.toml that does not parse, a flag it does not take).

Installed deps just work after htl pkg install. They go under .htl/modules/, beside the check cache; the installer is mlua-pkg's library. Inside, entries/<name> is where require("<name>") looks (src/<name> for a library htl new --lib wrote). Vendored is kept for the other thing: a copy of a dependency committed to the repo, which a target_dir entry in the manifest declares. When a directory is given, check / fmt / build / test do not enter target/, node_modules/, .mlua-pkgs/, any dot-directory, or a target_dir copy, which mlua-pkg install rewrites. A .tl that belongs to no module of the project is named once, with where it goes: 1 file(s) belong to no module of the project and were not checked: stray.tl; move them under src/, or set [layout] source = "." if the project root is where its modules are. A project that keeps its .tl at the root with no source directory is refused: htl: 3 file(s) at the project root belong to no module (lib.tl, lib_test.tl, main.tl): add [layout] source = "." to htl.toml, or move them under src/, exit 2. A type error in a dependency is reported with the dependency's own path and the file that required it —

error: .htl/modules/entries/mathx/init.tl:12:8: in local declaration: got string, expected number
  (required by src/geometry.tl)

htl check <file> works on a file in no project; htl check <dir> with neither an htl.toml nor an mlua-pkg.toml above the directory is an error that says what it looked for — a directory is checked as a project, and htl init makes it one.

A failure at run time names Teal, not the Lua htl generated — the file and the line that raised, and the same for every frame that reached it:

runtime error: ./depth.tl:8: attempt to index a nil value (local 'c')
stack traceback:
	[C]: in metamethod 'index'
	./depth.tl:8: in function 'depth.field'
	./depth.tl:12: in function 'depth.describe'
	boom.tl:3: in main chunk

Stripped bytecode — a bundle without --debug, include_tl_bytes! — has no frames to give; the .tl is still there, and htl run src/main.tl has them. A host whose users did not write the Teal shows them htl::user_message(&err) instead (developer_message).

Caching

htl check stores what it worked out under .htl/cache/ at the project root and replays what still holds; the summary says how much ([cached], [36/48 cached]). htl build, htl test and the macros read the same store. --no-cache turns it off and --explain-cache (HTL_CACHE_DEBUG=1) says why a lookup missed; --cache-mode / [cache] mode picks per-module (the default) or whole-run; htl cache status and htl cache clear look after the store. What makes an entry and what invalidates it: htl::cache.

Embedding in Rust

use htl::{Htl, TealRecord, host_module, include_tl, include_tl_bytes};

#[derive(TealRecord, Clone)]           // Teal record <-> plain table (IntoLua / FromLua)
pub struct Point { pub x: f64, pub y: f64 }

pub struct Host { started: std::time::Instant }

#[host_module(name = "host", dts = "scripts/host.d.tl", records = [Point])]
impl Host {
    pub fn uptime_ms(&self) -> u64 { self.started.elapsed().as_millis() as u64 }
    pub fn scale(&self, p: Point, k: f64) -> Point { Point { x: p.x * k, y: p.y * k } }
    pub fn greet(name: &str) -> String { format!("hello, {name}") }            // static
    pub fn parse(s: &str) -> Result<i64, std::num::ParseIntError> { s.parse() } // Err -> Lua error
}

const MAIN: &str = include_tl!("scripts/main.tl");              // checked at cargo build
const UTIL: &[u8] = include_tl_bytes!("scripts/util.tl");       // same, as stripped bytecode

fn main() -> anyhow::Result<()> {
    let h = Htl::new()?;
    Host { started: std::time::Instant::now() }.htl_preload(&h)?;
    h.preload_bytes("util", UTIL)?;
    h.exec(MAIN, "@scripts/main.tl", &[])?;         // frames read scripts/main.tl:<line>
    Ok(())
}

A type the host already sends as JSON is written once, with serde, and crosses as serde names it: TealRecord reads rename_all, rename, alias, skip, default, flatten, transparent, tag and the rest, so the .d.tl declares the keys serde_json writes and the table Teal receives carries them. A word it cannot honour stops the build naming the way out — #[teal(as = "..")], #[teal(serde)] or #[teal(skip)] on the field — and a data enum, which serde would write externally tagged, says #[serde(tag = "kind")] to agree. A host with no serde of its own derives through htl::serde and writes #[serde(crate = "htl::serde")]; the Call of examples/embed's --ask mode is one. What is read, passed through and refused: htl::dts.

The reference is docs.rs/htl: its front page walks this example, and the pages behind it are

  • host_module — parameters, errors = "return", Option, UserDataRef, uses, async fn (feature async; who runs the program that calls one is under Async);
  • dts — what each Rust shape becomes on the Teal side (#[derive(TealRecord)], serde's attributes, data enums as unions);
  • Strict<T> and Htl::strict_strings — the string boundary at run time;
  • Htl::with_checker_lua — a host running Teal it did not write builds the Lua itself;
  • dep_dts — shipping a crate's .d.tl to its users ([package.metadata.htl] dts), and what htl dts reports;
  • Htl::apply_project — publishing a crate whose Teal has dependencies (htl pkg patch, committed);
  • Project::for_host and TealResolver — a host that serves modules at run time, mods and plugins included (contract_resolvers, expect_type);
  • cexport and ffi — #[c_export], a C ABI for a caller that is not Rust (feature ffi; htl new --lib --target cdylib);
  • htl-mq — a window (htl new --target window);
  • htl-std — asynchronous file and process I/O (require("std.fs"), require("std.proc")).

Both ways of holding Teal are in this repository, built and run on every commit: examples/ has embed, where include_tl!, include_bundle!, #[derive(TealRecord)] and #[host_module] all meet in one binary (and, with --async, an async fn awaited from Teal), and resolver, where nothing is embedded and require goes through mlua-pkg at run time. Its README says what each one prints and which line of the output is the point.

Calling Teal from async Rust (htl::run, [rust] exports)

This is the boundary Teal owns (the intro's other direction): Teal declares the functions and their types, and the Rust that calls them is generated from those declarations.

A host whose logic is in Teal starts one runner and calls the functions of a module on it. With htl = { features = ["async"] }, [rust] exports = ["app"] in htl.toml, and htl::teal_types!(); at the crate root (or htl rs, which writes the same code to a file), the record a module returns becomes a typed client; an argument or a result that disagrees with the Teal declaration fails cargo build:

local record app
   add: function(a: integer, b: integer): integer   -- a function the body declares is a method
end

function app.add(a: integer, b: integer): integer
   return a + b
end

return app
htl::teal_types!();                                // at the crate root: reads `[rust]` of htl.toml

async fn run() -> anyhow::Result<()> {
    let (runner, driver) = htl::run::start(htl::run::RunnerConfig {
        search_paths: vec!["src".into()],
        deadline: std::time::Duration::from_secs(5),
        queue: 64,
    })
    .await?;
    let app = app::App::load(&runner, &[]).await?; // `require("app")`, in a state of its own
    let n: i64 = app.add(2, 3).await?;             // Result<i64, htl::run::CallError>
    driver.shutdown().await?;
    Ok(())
}

Every call runs under the runner's deadline and is cut off past it, within the limits htl::run states. Each load is an instance with its own program state, so two instances of one module share nothing of the module's own. A client is named for the record its module returns, so the clients of local record M modules are all M, told apart by their module path (shop::app::M). What a runner promises is htl::run; which functions get a method, how each type is written, and what is refused is the Exports section of htl::rsgen.

Lints (htl check, include_tl!)

Every rule has a level — allow, warn or deny — and a project sets it by name:

[lint.rules]
nil-index = "deny"        # this one stops the run
no-any = "warn"           # allow by default; see it while you migrate, without failing CI
"tl:hint" = "allow"       # quote a name with a `:` — TOML has no bare key for it

The same names go on the command line (--lint nil-index=deny,no-any=warn,-tl:hint) and in HTL_LINTS; [lint] strict / --strict make every warn count as deny; a trailing -- htl: allow(nil-index) silences one occurrence. htl check --list-lints prints every rule with its default level, htl check --explain <rule> prints why a rule exists, what the fix is and the judgment call where there is one (a run with findings names the flag once per rule, before its summary), and htl::lint is the reference: what each rule catches, Teal's own warnings under tl:*, and the six markers (---@struct, ---@sealed, ---@extensible, ---@nilable, ---@async, ---@noyield, which #[host_module] writes for a sync fn's Function parameters and #[derive(TealRecord)] for a field marked #[teal(noyield)]) that turn a rule on for a record or a function (the full set of nine is what htl adopt reports; see Adoption).

Project config (htl.toml)

Every command that reads the project, and include_tl!, read the htl.toml at the project root; flags and HTL_LINTS / HTL_LINT override it, and htl new writes a commented one. The keys, with what each is for:

[toolchain]
htl = "0.19"              # the htl command this project expects; a mismatch is refused

[lint]
strict = true             # every warn counts as deny

[lint.rules]
nil-index = "deny"

[layout]                  # where this project's own files live: source / types / tests
source = "src"            # "." for a flat project

[check]
paths = ["mods"]          # extra dirs require() resolves from while checking

[imports]
mathx = "dep:mathx"       # a name the project and a dependency share: which one it means

[build]
target = "bin"            # what runs this project's output: hb (default), bin, cdylib, window

[[contract]]              # where this project accepts modules written outside it
dir = "mods"

[async]                   # see Async: a cancelled program's time to clean up, and preemption
grace_ms = 1000
# preempt = 1

[lang]
async = true              # async / await are keywords and the five rules are on (see Async)

[rust]                    # htl rs / teal_types!: Rust from this project's Teal (0.19 and later)
modules = ["ev"]          # records and enums of these modules become Rust types
exports = ["app"]         # these modules' declared functions get a typed client (feature async)

[rust.types]
"ev.Event.at" = "u32"     # a field's Rust type, in place of the lowered one

htl::config has the full sample and every key ([fmt], [fix] and [cache] among the ones not shown here); htl::model is how a file gets its name, which module owns it, and what a dependency may see; htl::contract is ---@contract / ---@required, the shape a directory of modules from outside the program must have, and what htl publishes for their authors.

Which file a name resolves to (htl resolve)

$ htl resolve mq
htl resolve mq: src/mq.d.tl

  order  file                  kind         status
  1      src/mq.d.tl           declaration  read
  2      types/mq.d.tl         declaration  shadowed by 1
  3      types/htl-mq/mq.d.tl  declaration  shadowed by 1  (shipped by htl-mq 0.19.0)

  answered by the project model

Every status, the host-provided and environment-provided answers, and the JSON form: htl::resolve.

Patched dependencies (htl pkg patch)

htl pkg patch mathx copies the dependency's package root into patches/mathx/, writes patch_dir onto it in mlua-pkg.toml, and records the revision as patch_base; from there the directory is the project's code, and htl check walks it. What is dropped, what happens when the pin moves, and --force: MluaProject::patch.

The native modules (std.*)

local json = require("std.json")         -- typed via std/json.d.tl, inside the binary
local rows: {Row} = json.decode(text)    -- decode is generic: annotate the result

json, env, path, time, string, validate, pretty and argparse, from mlua-batteries; every function raises, so a result-style call is pcall. fs and proc are htl-std's, and asynchronous only — local fs = require("std.fs") then await fs.read(p); a select (Async, below) can wait on a process and a timer together. htl run and htl test have all of it with no host to ask for it; a Rust host adds htl_std::install(&h)? beside h.install_std()? (feature std, on by default) in preload. The rest is htl::batteries and the htl-std crate doc.

Tests

local t = require("htl.test")            -- typed via test.d.tl
t.describe("util.add", function()
   t.it("adds", function()
      t.expect(util.add({x=1,y=2}, {x=10,y=20})):to_equal({x=11,y=22})
   end)
end)

A test file is any .tl that loads the test library, wherever it is; htl test runs each in a state of its own, on the executor (Async) — a test may call a host's async fn as it calls any other. Matchers, snapshots, the seeded t.rng(), --coverage / --lcov / --junit, and the --lib contract for another library: htl::testing.

Editors (htl lsp)

htl lsp is a language server on stdio, and what it reports is what htl check reports for the file: the project's resolution rules, lint levels, -- htl: allow(..) comments and .d.tl declarations, with the same messages and lines. No extension is shipped; a client that can start a language server by command starts it as htl lsp.

Neovim (0.11 or later), in init.lua:

vim.lsp.config('htl', {
  cmd = { 'htl', 'lsp' },
  filetypes = { 'teal' },
  root_markers = { 'htl.toml', 'mlua-pkg.toml' },
})
vim.lsp.enable('htl')

Helix, in languages.toml:

[language-server.htl]
command = "htl"
args = ["lsp"]

[[language]]
name = "teal"
scope = "source.teal"
file-types = ["tl"]
roots = ["htl.toml", "mlua-pkg.toml"]
comment-token = "--"
language-servers = ["htl"]

VS Code has no htl extension; a generic language-client extension that runs a command as a server can be pointed at htl lsp for .tl files.

It publishes diagnostics when a file is opened or saved, formats, offers htl fix's fixes as quick fixes and a fix-all action, answers hover and go-to-definition, lists the symbols a file declares, searches a symbol by name across the project, finds references and traces call hierarchies. It does not yet check while typing, complete or rename.

When a file is checked, what a diagnostic becomes, which file changes re-check the open ones and what the server leaves to htl check are in htl lsp --help and the module doc of crates/htl-cli/src/lsp.rs.

Async

The executor

htl run and htl test run a program as a root coroutine on an executor (mlua-isle's, on tokio), and so does a host that awaits Htl::run_async on a runtime of its own or calls Htl::run_blocking (feature async). Inside it a call of a host's async fn suspends the program and resumes it with the value, as any other call; a plain Htl::exec has nothing to suspend to, and such a call raises there. Ctrl-C in htl run cancels the root: the program gets its grace to clean up (Settings, below), htl run: interrupted is printed and the exit code is 130. The embed example (cargo run -p embed -- --async) is a host with such a method. Besides a host's async fn, a task, a channel or a timer, what a program awaits includes std.fs and std.proc (std.*, above).

A host that calls the functions of a Teal module from its own async code uses htl::run instead (Calling Teal from async Rust, above). That is not this executor's setup: the module lives on a VM thread of the runner's, in a bare safe state (no debug, no C modules, no host or std.* modules), and a call cut at the runner's deadline gets no grace, whatever [async] grace_ms says.

async / await

With [lang] async = true in htl.toml, a program says which functions may suspend and where, and the checker holds it to that. Teal is not forked: the two words are keywords only for a project that says so (a Teal language server still reports them as errors), and htl fmt leaves them where they are.

local http = require("http")                    -- a #[host_module] with async fn

local async function pair(a: string, b: string): string, string
   async local x = http.get(a)                  -- a child task; x: Task<string>
   local y = await http.get(b)                  -- the marker on a call of an async function
   return await x, y                            -- the task's value; leaving without it cancels the task
end

print(await pair("/a", "/b"))                   -- the entry chunk of htl run / htl test is async
written meaning generated Lua, on the same line
local async function f(..), global async function f, async function R.f(..), async function(..) f may suspend; its type is unchanged the function without the word
await f(x) the marker on a call of an async function; binds like a prefix (await f(b) + 1 awaits f(b)) f(x)
async local x = e runs e as a child task; x: Task<T>, T the type of e local x <close> = require("htl.task").spawn(function() return e end)
await x, x an async local the task's value; what it raised is re-raised x:await()

One name per async local (a task holds one value), and the expression is what runs as the task — a call of an async function, usually, since a plain value has nothing to wait for. The checker types x from e (it reads the declaration as require("htl.task").of(e)) where the closure the generated Lua spawns would tell it nothing. A task is awaited once; await x twice is htl.task: task already awaited at run time. After . or : and before : = , ) . ( the words are still names (t.await, x:await(), a field await:, { await = 1 }), so a file that used them as names keeps working with the setting on.

The rules

Five rules, on by default with the setting and silent without it (htl check --list-lints shows the level of each; the levels change like any other rule's, Lints):

rule level reported
await-missing deny a call of an async function — a Teal async function, or a host method whose declaration line carries ---@async (what #[host_module] and htl dts write for an async fn) — without await, at the call
await-outside-async deny await or async local inside a function that is not async, at the keyword; and at the top level of a module reached through require, which cannot yield
await-non-async warn await on a call of a function that is not async, at the keyword
task-escape deny an async local name captured by a nested function or returned bare: the task is cancelled when its scope ends, so what the capture or the caller reads is a cancelled task
async-as-sync-callback deny an async function handed to a callee that calls it from C — table.sort's comparator, string.gsub's replacement, xpcall's message handler, a __tostring in a table constructor, a host parameter or record field its .d.tl marks ---@noyield — where a suspension fails (or, in the handler, is lost), at the argument

The top level of the file being checked is the entry chunk of htl run / htl test and is async; a module's top level is not, and that is reported on the check of each file that requires the module, at the module's line — so htl check src says something about a module's top-level await once a checked file requires it. The expression of an async local runs inside the task and needs no await of its own.

Tasks (htl.task)

The syntax desugars to this module, and a program can use it directly — with the setting off, or for a task that is not one async local:

local task = require("htl.task")          -- typed via task.d.tl
local a = task.spawn(function(): string return http.get("/a") end)
local b = task.spawn(function(): string return http.get("/b") end)
print(a:await(), b:await())              -- 300 ms for two 300 ms calls

spawn starts the closure as a task beside the caller and returns a Task<T>, T being the closure's declared return type (an untyped closure is a type error; a task holds one value). t:await() waits for it and hands the value back, or re-raises what the task raised, as it was; t:cancel() asks a task to stop, t:done() says whether it has. Tasks are structured: a task the parent did not await is cancelled and waited for when the parent returns, raises or is cancelled, and local t <close> = task.spawn(..) does the same at the end of the scope. A cancel reaches a task at its next await as an error that task.is_cancelled recognises, so a pcall or a <close> handler can clean up and let it pass. Under [lang] async the closure is written async function(): string return await http.get("/a") end: a call of an async function inside a plain closure is await-missing there. The declaration's header is the reference: htl::task.

The module also has channels, timers and select, so a loop in Teal can wait on whichever source is ready first — events a Rust host pushes, values from other tasks, a tick, a timeout:

local events = daemon:events()            -- task.RecvChannel<daemon.Event>, from the host
local tick = task.ticker(1000)
local stop = await task.select({
   events:on(function(ev: daemon.Event, ok: boolean): boolean return not ok end),
   tick:on(function(_ms: number, _ok: boolean): boolean return false end),
   task.after(5000):on(function(): boolean return true end),
})

Each case's handler is typed by its source, and every case of one select returns the same type; send, recv, wait and select are reported by await-missing when written without await. On the Rust side, a #[host_module] method returning htl::task::RecvChannel<Event> or SendChannel<Report> is declared as task.RecvChannel<Event> / task.SendChannel<Report>, and the generated .d.tl imports htl.task for it, so send on a receive-only channel or a handler of the wrong type fails include_tl! at cargo build. How a host makes the channels: htl::task.

A host asks Teal through htl::task::Ask<Q, A>, one kind of question with the kind's deadline — there is no library default. Ask::new hands back two ends: the Ask, which the host struct holds and whose getter gives Teal the channel of task.Request<Q, A> to answer with req:reply, and the Asker, Send + Clone, for a handler on another thread or runtime to call asker.ask(q).await on. The answer is Ok, or an AskError: Timeout (a late reply returns false), Closed (the loop is gone), NoReply (taken and left unanswered) or Rejected(reason) (Teal refused with req:reject(reason), the reason a string). The host maps any Err to its safe side in one place, and dropping the last Asker ends the loop. The --ask mode of examples/embed is one, end to end: htl::task.

Settings

Two tables of htl.toml, both with every key in htl::config:

  • [lang] async = true makes the two words keywords and turns the rules on (include_tl! reads it too). Off, the default, they are names, and htl.task is still there.
  • [async] grace_ms (default 1000) is how long a cancelled program — the root, and every task under it, on one deadline — may keep running its cleanup. With 0 it is dropped at once: the host's future is released, its __close handlers run but cannot await. With more, the cancel arrives as an error first, so a <close> handler or a pcall sees it and may await a host call, and the drop comes when the grace ends. A host async fn returns the cancel at its own await rather than waiting the grace out. [async] preempt is off: a task runs from one await to the next without another task interleaving, which is what requiring the word is for; the price is that a task in a CPU loop cannot be cancelled by a sibling on the same runtime, only from another thread (Ctrl-C, an Interrupt). preempt = N yields it every N cancel checks (a check every 1000 instructions), and then tasks interleave at points the program did not mark.

htl gen writes plain Lua for all of this, and that Lua requires htl.task: it runs on a host that preloads the module, which is the htl crate with the async feature — mlua-isle on the host — running the program on the executor, and nowhere else.

Fixing (htl fix)

Some diagnostics carry a mechanical fix; htl check marks them (fixable: htl fix) and htl fix [paths] applies the safe ones (--unsafe for the rest; suggest is shown under --diff and never written). Which rules carry a fix, the [fix] keys, and the flags: htl::fix.

Machine-readable output

--format json on check, test, fix, unused, adopt, resolve, cache status and bundle info prints one JSON document on stdout, with the same exit code as the text form. check --statistics adds a statistics array to that document (and diagnostics is then empty) — the same table --statistics prints to stderr otherwise. The shapes: htl_cli::report, htl::unused, htl::adopt, htl::resolve. The shapes of fix, cache status and bundle info are not documented yet.

Bundles (htl build)

htl build src/main.tl -o app.hb follows require("<literal>") from the entry and links everything it reaches into one file of stripped bytecode (--debug, --source); htl run app.hb runs it, htl bundle info app.hb says what it holds, and a Rust host embeds the same thing with include_bundle!. Host-provided names, portability, the store: htl::link and htl::bundle.

Unused (htl unused)

htl unused reports a module nobody requires and a dependency declared and never used, walked from the entries the project already declares (main.tl, tests, contracts, [build] extra, what a Rust host embeds); exit 0 unless --exit-non-zero-on-unused. htl::unused.

Adoption (htl adopt)

htl adopt counts how many declarations carry each of htl's nine markers (---@struct, ---@optional, ---@sealed, ---@extensible, ---@nilable, ---@contract, ---@required, ---@async, ---@noyield) and, with --detail, where: htl adopt: 3 of 9 features used, 4 markers in 2 files. applicable — how many declarations the evidence in the project's own code says could carry a marker — is empty until a lint supplies it; ---@struct's and ---@sealed's rows have one so far (unmarked-struct, unmarked-sealed), so every other row's is still empty. A report, not a gate: the exit code is always 0. htl::adopt.

Layout of a project (htl new)

<name>/
├── mlua-pkg.toml          [package] entry = "src/<mod>"  → consumers require("<name>")
├── htl.toml               [fmt], and commented [lint] / [check] / [[contract]] / [async] /
│                          [lang] to fill in;
│                          [build] target with --target. Read by CLI and macro
├── mise.toml              "cargo:htl-cli" = "<this CLI>": the command, for mise
│                          (written by a released CLI, not by a checkout build)
├── .gitignore             .htl/ (the cache and the installed deps) and *.hb; a target
│                          adds its own lines
├── README.md              the commands to run, and what the manifest's entry is for
├── src/<mod>/init.tl      the module (require("<mod>") from src/ and tests/)
├── types/README.md        .d.tl the project consumes (hand-written, and <crate>/ copied
│                          from a dependency) and publishes (a ---@contract type) go here
├── patches/<dep>/         a dependency taken into the tree (htl pkg patch), committed;
│                          checked, not formatted, its tests not run
├── src/main.tl            entry script (left out with --lib)
└── tests/<mod>_test.tl     a test (it loads htl.test); helpers beside it are not run

mlua-pkg.toml names one dependency from the start, htlx (htl-x, pure Teal collections), so the first step is htl pkg install; --no-x leaves it out. --target bin | cdylib | window adds a Rust host (--embed is --target bin; htl init --target for an existing project), and the htl it pins is the one the binary was built with (--htl main, --htl path:<checkout> to pin another one; htl pin moves an existing project between them afterwards). What each target produces and what the scaffold writes for it: BuildTarget and htl-cli.

Unions of records (where)

Teal refuses a union of two record types on its own; a where self.kind == "…" clause on each record lifts it, is narrows, and union-exhaustive reports a chain that leaves a variant untested (htl::lint). A record cannot open with a field called where; ["where"]: <type> works anywhere.

Pitfalls the checker now names

  • Case-insensitive filesystems (macOS, Windows): require("site") from a file called Site.tl resolves to that very file; htl says so, and that one of the names has to change.
  • Numeric inference: local n = 0 is integer, 0.0 is number; opt into the explicit-number lint to be told where an annotation is missing.
  • Forward references: function world.tick calling world.observe that is defined further down is "invalid key 'observe' in record 'world'"; htl names the later definition and hands over the line to paste into the record (observe: function(w: World, what: string)).
  • A union of two records: "cannot discriminate a union between multiple table types" reads like a limit on the type system, and it is a limit on is, which each record can lift for itself with a where clause (see above).
  • A field called where: only on a record or interface body's first line, where Teal reads the union predicate before the fields. The bare "syntax error" and the follow-on error on the next field's line become one message naming the keyword and the two ways out — ["where"]: <type>, or any other field first (see above).
  • Multi-value call in last position: t.expect(can_cast(x)) with can_cast returning boolean, string is a 2-argument call; htl names the expanding call and the two fixes (bind first, or parenthesize to keep the first value).

Running against an unpublished htl

A project htl new wrote moves with htl pin path:<checkout> (and back with htl pin release, once that checkout's release is out) — it rewrites the same htl / htl-mq line itself, mise.toml and [toolchain] htl with it, and runs the cargo update that makes Cargo.lock agree.

Every other consumer — anything not an htl new tree, or an htl line htl pin does not recognise — points at a checkout by hand, in its own Cargo.toml:

[patch.crates-io]
htl = { path = "/path/to/htl/crates/htl" }
htl-core = { path = "/path/to/htl/crates/htl-core" }
htl-macros = { path = "/path/to/htl/crates/htl-macros" }

All three, not one; then cargo update -p htl -p htl-core -p htl-macros once, and the same command again with the block deleted to go back.

What is deliberately not here

  • No Teal fork: tl.lua is vendored verbatim (0.24.8, MIT).
  • No token-level formatting: htl fmt recomputes indentation and whitespace only.
  • No Luau: PUC Lua 5.4 through mlua, and no dual bytecode-plus-source payload (the Portability section of htl::bundle).
  • .d.tl files are written syntactically, from Rust source and from Teal (the module a ---@contract type is declared in), not by reflection on the types. What does read the checker's types is the other generation: htl shapes lowers Teal records and enums to an lshape shape, and htl rs (and teal_types!) to Rust types and, for a module in [rust] exports, its declared functions to a typed client — from what a check knows of the file.

License

MIT OR Apache-2.0. Teal (crates/htl-core/vendor/tl.lua) is MIT, see crates/htl-core/vendor/LICENSE.teal.

About

Holistic Typed Lua: Teal on mlua, hidden behind cargo

Resources

Readme

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

2 forks
Report repository

Releases

Packages

Contributors

Languages
0