8000 Schema module: one validation library for JSON and block-syntax data files · Issue #650 · kexhq/kex · GitHub
[go: up one dir, main page]

Skip to content

Schema module: one validation library for JSON and block-syntax data files #650

Open

Description

@akoskovacs

Idea

A Schema module in the standard library: parse data into a typed value, and report every problem with its path. JSON and block-syntax data files are two inputs to the same schema.

The second input is the newer part. Some .kex files are pure data written in block syntax, with no logic. package.kex is the existing example: docs/plan_tey_pm.md describes it as "parsed as data and never evaluated", and Tey has its own reader that knows which entries are allowed. A small block-file reader would turn such a file into data under a schema's guidance, without evaluating it, so any tool can define a file format and get checking for free.

Sketch

Kex has types at run time, so a record already describes the structure. The schema starts from the record and adds only what the record cannot say.

record Package do
  name     : String
  version  : String
  license  : String?
  commands : [Command]
  authors  : [String]
end

let packageSchema = Schema.of(Package)
  .field(:version, matching: regex`^\d+\.\d+\.\d+`)
  .field(:authors, nonEmpty: true)

packageSchema.parse(json)              # from JSON
packageSchema.load("package.kex")      # from a block file

A schema-first form in a compiled block that generates the record is a possible second entry point, for people who prefer to write the schema and derive the type.

What it covers

Need Expressed as
Exactly one a plain field (String)
Zero or one an optional field (String?)
Zero or more a list field ([Command])
One or more a list field with a non-empty rule
Defaults the record's field defaults
Rules beyond the type pattern, range and length rules on a field
Result Result<T, [Issue]>, each issue with its path and, for a file, its line

The same schemas apply to environment variables and request bodies.

Block files

What the reader adds on top of the schema:

  • entry names, and how they map to fields;
  • positional and named arguments, as in command("specs", run: "make spec");
  • repeated entries collected into a list field;
  • the block's own argument, as in package "kex" do.
package "kex" do
  version("0.4.0-rc.3")
  license("MIT")
  author("...")
  command("specs", run: "make spec", description: "Run the spec suite")
end

The file is walked as data and never run, so a manifest from someone else's package cannot execute code. A file with no logic has no if, loop or call around its entries, so the counts are known when it is read and the errors can be exact: "version is given twice (lines 4 and 9)", "unknown entry licence".

Completion and hover in a data file can come from the schema.

Versioning

A manifest published today has to stay readable by every later tool, and an older tool has to fail clearly on a newer file.

  • The file states its version in a form that never changes: package "kex", schema: 3 do. A file with no version is version 1, so existing files stay valid.
  • This is separate from kex(">= ...") in package.kex, which says which toolchain builds the package.
  • Entries are marked with the versions they apply to (since:, until:, renamedTo:) in one definition, so every old version stays readable and the history is visible in one place.
  • The loader upgrades an older file to the latest shape, so a program only handles the newest record. A rename is covered by renamedTo:; anything that needs computation is a migration step.
  • A newer file on an older tool reports "this file uses schema 4; this tool understands up to 3", not a list of unknown entries.
  • A deprecated entry still loads, with a warning that names its replacement.
Change New version needed
Add an optional or repeated entry no, but mark it since: so older schema versions reject it
Add a required entry yes
Remove or rename an entry yes
Make an entry's type or count stricter yes

Open questions

  1. How a file names its schema. By file name, by a line in the file, or by whoever loads it. The language server needs the answer to offer completion.
  2. What "no logic" allows. Literals, lists, maps and entries are clear; constants, string interpolation and references to other entries are the borderline cases.
  3. The surface of the API. The ideas follow Zod (parse into a typed value, collect all issues with paths, compose small schemas), but the calls should be Kex's own. NimbleOptions and Ecto changesets solve the same problem with definitions as data and are worth comparing.
  4. Entries with several arguments. command("specs", run: ..., description: ...) has to become a Command. Either the arguments are the fields of an existing record, or the entry takes a nested definition, which also gives nested sections.
  5. Embedded blocks. If the same schema generates entry functions for a DSL block inside a program, those collect entries when they run, so counts there are a run-time check unless the block is fully determined at compile time.
  6. Prerequisites. A parser entry point that returns a file as data, and reflection that can list a record's fields and construct the record, both usable from the standard library.

Earlier sketch

The request started as a DSL module with a builder:

DSL.define :package do |p|
  p.once(:name, String)
  p.once(:version, String)
  p.once(:license, String)
  p.multiple(:commands, [Command])
  p.oneOrMore(:author, String)
end

once, multiple and oneOrMore became ordinary rules on fields once the schema was shared with JSON, which leaves only the entry-to-field mapping as block-specific.

Alternatives considered along the way:

  • A new block type that fills a record, with counts taken from the field types. This needs a language change and cannot name entries differently from fields.
  • A T+ type for "one or more", as sugar beside T?. Useful on its own (first and max could return T instead of T?), but not required once the count is a schema rule.

Activity

  1. changed the title [-]DSL module: schema-checked data files in block syntax[/-] [+]Schema module: one validation library for JSON and block-syntax data files[/+] on Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0