Repository navigation
Description
Idea
A Schema module in the standard library: parse data into a typed value, and report every problem with its path. JSON and block-syntax data files are two inputs to the same schema.
The second input is the newer part. Some .kex files are pure data written in block syntax, with no logic. package.kex is the existing example: docs/plan_tey_pm.md describes it as "parsed as data and never evaluated", and Tey has its own reader that knows which entries are allowed. A small block-file reader would turn such a file into data under a schema's guidance, without evaluating it, so any tool can define a file format and get checking for free.
Sketch
Kex has types at run time, so a record already describes the structure. The schema starts from the record and adds only what the record cannot say.
record Package do
name : String
version : String
license : String?
commands : [Command]
authors : [String]
end
let packageSchema = Schema.of(Package)
.field(:version, matching: regex`^\d+\.\d+\.\d+`)
.field(:authors, nonEmpty: true)
packageSchema.parse(json) # from JSON
packageSchema.load("package.kex") # from a block file
A schema-first form in a compiled block that generates the record is a possible second entry point, for people who prefer to write the schema and derive the type.
What it covers
| Need | Expressed as |
|---|---|
| Exactly one | a plain field (String) |
| Zero or one | an optional field (String?) |
| Zero or more | a list field ([Command]) |
| One or more | a list field with a non-empty rule |
| Defaults | the record's field defaults |
| Rules beyond the type | pattern, range and length rules on a field |
| Result | Result<T, [Issue]>, each issue with its path and, for a file, its line |
The same schemas apply to environment variables and request bodies.
Block files
What the reader adds on top of the schema:
- entry names, and how they map to fields;
- positional and named arguments, as in
command("specs", run: "make spec"); - repeated entries collected into a list field;
- the block's own argument, as in
package "kex" do.
package "kex" do
version("0.4.0-rc.3")
license("MIT")
author("...")
command("specs", run: "make spec", description: "Run the spec suite")
end
The file is walked as data and never run, so a manifest from someone else's package cannot execute code. A file with no logic has no if, loop or call around its entries, so the counts are known when it is read and the errors can be exact: "version is given twice (lines 4 and 9)", "unknown entry licence".
Completion and hover in a data file can come from the schema.
Versioning
A manifest published today has to stay readable by every later tool, and an older tool has to fail clearly on a newer file.
- The file states its version in a form that never changes:
package "kex", schema: 3 do. A file with no version is version 1, so existing files stay valid. - This is separate from
kex(">= ...")inpackage.kex, which says which toolchain builds the package. - Entries are marked with the versions they apply to (
since:,until:,renamedTo:) in one definition, so every old version stays readable and the history is visible in one place. - The loader upgrades an older file to the latest shape, so a program only handles the newest record. A rename is covered by
renamedTo:; anything that needs computation is a migration step. - A newer file on an older tool reports "this file uses schema 4; this tool understands up to 3", not a list of unknown entries.
- A deprecated entry still loads, with a warning that names its replacement.
| Change | New version needed |
|---|---|
| Add an optional or repeated entry | no, but mark it since: so older schema versions reject it |
| Add a required entry | yes |
| Remove or rename an entry | yes |
| Make an entry's type or count stricter | yes |
Open questions
- How a file names its schema. By file name, by a line in the file, or by whoever loads it. The language server needs the answer to offer completion.
- What "no logic" allows. Literals, lists, maps and entries are clear; constants, string interpolation and references to other entries are the borderline cases.
- The surface of the API. The ideas follow Zod (parse into a typed value, collect all issues with paths, compose small schemas), but the calls should be Kex's own. NimbleOptions and Ecto changesets solve the same problem with definitions as data and are worth comparing.
- Entries with several arguments.
command("specs", run: ..., description: ...)has to become aCommand. Either the arguments are the fields of an existing record, or the entry takes a nested definition, which also gives nested sections. - Embedded blocks. If the same schema generates entry functions for a DSL block inside a program, those collect entries when they run, so counts there are a run-time check unless the block is fully determined at compile time.
- Prerequisites. A parser entry point that returns a file as data, and reflection that can list a record's fields and construct the record, both usable from the standard library.
Earlier sketch
The request started as a DSL module with a builder:
DSL.define :package do |p|
p.once(:name, String)
p.once(:version, String)
p.once(:license, String)
p.multiple(:commands, [Command])
p.oneOrMore(:author, String)
end
once, multiple and oneOrMore became ordinary rules on fields once the schema was shared with JSON, which leaves only the entry-to-field mapping as block-specific.
Alternatives considered along the way:
- A new block type that fills a record, with counts taken from the field types. This needs a language change and cannot name entries differently from fields.
- A
T+type for "one or more", as sugar besideT?. Useful on its own (firstandmaxcould returnTinstead ofT?), but not required once the count is a schema rule.
Activity
[-]DSL module: schema-checked data files in block syntax[/-][+]Schema module: one validation library for JSON and block-syntax data files[/+]