8000 fix: type whole rows, null elements and json numbers soundly by colinhacks · Pull Request #5 · colinhacks/reylo · GitHub
[go: up one dir, main page]

Skip to content

fix: type whole rows, null elements and json numbers soundly - #5

Merged
colinhacks merged 4 commits into
mainfrom
json-soundness
Oct 22, 2026
Merged

colinhacks merged 4 commits into
mainfrom
json-soundness

Conversation

@colinhacks
@colinhacks colinhacks commented Oct 22, 2026 •
Copy link
Copy Markdown
Owner

Three types inside JSON a statement builds were unsound, one was looser than the statement shows, and one shape of result was silently wrong. Each was found by running SafeQL's generator test statements through the soundness oracle, which also gained the check that would have caught them.

Statement Typed before What Postgres returns Typed now
select jsonb_agg(team.*) from team number[] | null [{"id": 1, "name": "core"}] { id: number; name: string }[] | null
select json_build_object('v', array[name]) from team (name is not null) { v: (string | null)[] } {"v": ["core"]} { v: string[] }
select jsonb_build_object('n', 'NaN'::numeric) { n: number } {"n": "NaN"} { n: number | "NaN" | "Infinity" | "-Infinity" }
select jsonb_agg(s) from (select id from team) s (string | null)[] | null [{"id": 1}] (Record<string, unknown> | null)[] | null
select m.first_name as x, mt.id as x from … { x: string } {"x": null} (drivers keep the last column) error reylo14 at the second column
  • t.* inside a call is the whole row. jsonb_agg(t.*), json_agg(t.*) filter (…), jsonb_build_object('t', t.*) and to_jsonb(t.*) read the row, as the bare t already did. A row with a column the probe proved non-null is no longer nullable under an outer join, since an unmatched row is every column NULL.
  • An array built inside JSON keeps what the statement shows of its elements, as a result column's does: array[name] over a not null column is string[], and array[name, null] stays (string | null)[]. (When this PR was opened, array[name, null] inside JSON was typed string[]; fix(postgres): type returned array elements as nullable #12 has since made every array's elements nullable unless the statement shows otherwise, so what this PR adds there is the narrowing.)
  • Non-finite numbers and unread rows inside JSON. Postgres writes a non-finite float4, float8 or numeric inside a document as text, and a row whose columns were not read as an object. The TypeScript types, .outputSchema and the engine's JSON Schema say so; MySQL, which stores no infinity, keeps number.
  • Two result columns of one name are an error (reylo14) on all three databases, at the second column on Postgres and on the statement's first word elsewhere, and the row type keeps one key, the last, as drivers and JSON.parse do.
  • The oracle checks each value's kind inside a JSON document (a number, text, a boolean, an object, an enum's label, each array element), not only its nulls. New case oracle/cases/json_whole_rows; four existing cases re-blessed for the new error and one more precise row.
  • Docs: the Postgres page on values inside a document, the errors section, and the captured hover types.

Verified locally: the engine's native tests, all 98 oracle cases on Postgres 18.6, SQLite and MySQL 8.4 with --strict-coverage, the three rebuilt engine.wasm artifacts, nub run typecheck, the loader's tests with every lab installed, the package build and the docs site.

Copilot AI balanced review requested due to automatic review settings October 22, 2026 16:07
@pullfrog
pullfrog Bot commented Oct 22, 2026 •
Copy link
Copy Markdown
Contributor

No configured credential can serve this run. Your Claude Pro/Max subscription was rejected. Sign in again by running pullfrog auth claude in a terminal.

The providers' responses
CLAUDE_CODE_OAUTH_TOKEN: 401: OAuth access token has been revoked.

Pick another model →

Pullfrog  | Rerun failed job ➔ | View workflow run | via Pullfrog | Using claude-opus-5-5

@vercel
vercel Bot commented Oct 22, 2026 •
Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
reylo Ready Ready Preview Oct 22, 2026 11:42pm UTC

Request Review

Copilot AI left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

8000

@pullfrog pullfrog Bot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No correctness issues found. One small doc-comment slip is noted inline.

Reviewed changes

Pullfrog hadn't reviewed this PR before, so this pass covers the whole of ea2a9e7. The only difference from 1e19a29 is that the branch was rebased onto main.

  • Treated t.* in a call as the whole row. jsonb_agg(t.*), json_agg(t.*) filter (…), jsonb_build_object('t', t.*) and to_jsonb(t.*) now go through the new Builder::whole, the same path the bare t uses. leaf now refuses any star, so a star never ends up in a probe's select list.
  • Narrowed whole-row nullability under outer joins. A row is now non-null when the probe proves any of its columns non-null. This is sound because probe columns get the same outer-join nulling from describe (by name, or by position via item_columns) that result columns get.
  • Kept NULL elements of a statement-built array inside JSON. ResolveCtx::column now applies with_items from the probe's analysis.
  • Typed non-finite numbers and unread rows inside documents. Inside JSON, Postgres float4/float8/numeric now admit "NaN", "Infinity" and "-Infinity", and MySQL keeps number. A record is now an object.
  • Made duplicate result column names error reylo14. The new analysis::clashes check runs on all three engines. On Postgres, column_spans places the error on the alias or column name; on MySQL and SQLite it lands on the statement's first word. On the TS side, lastOfEachName and object() keep the last column of each name, which matches what drivers and JSON.parse do.
  • Made the oracle check value kinds inside documents. json_contradiction now walks array elements and checks scalars (via json_admits) and enum labels.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread src/render.ts Resolved
@colinhacks
colinhacks force-pushed the json-soundness branch 2 times, most recently from 95de6c2 to 0e78008 Compare October 22, 2026 18:05
@pullfrog pullfrog Bot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No correctness issues from the rebase. One comment went stale under #12 and is noted inline.

Reviewed changes

Since the last review at 95de6c2, the branch was rebased onto 7de25a3 (#12, returned array elements nullable). git range-diff shows the code patches are unchanged. Only the context of the site/lib/sql-modules.json hunk moved.

  • Rebased onto #12. All three engine.wasm blobs differ from both the new base and the previous head, so they were rebuilt on top of #12's engine.
  • Checked how the PR combines with #12. An array whose nullable_items is absent now renders (T | null)[] inside JSON too. So with_items in ResolveCtx::column now mostly narrows: an array proven non-null (array_non_null.sql → (text!)[]) renders T[]. The oracle's new in-JSON element check (nullable_items == Some(false)) matches that rule. The PR's schema.rs, render.ts and schema.ts hunks (non-finite numbers, record, lastOfEachName) don't overlap #12's array changes in those files.

ℹ️ The PR description's "typed before" for array[name, null] no longer holds against main

Since #12, main already types json_build_object('v', array[name, null]) as { v: (string | null)[] }. The bullet "An array the statement builds keeps its NULL elements inside JSON" now describes behavior main already has. What this PR still adds there is the narrowing to T[] where the probe proves the elements non-null. Worth updating before squash-merge if the description becomes the commit message.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread oracle/src/seed.rs Resolved
@pullfrog pullfrog Bot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. The rebase onto #15 is clean, and the open seed.rs comment is fixed.

Reviewed changes

Since the last review at 0e78008, the branch was rebased several times, most recently onto 3cfd016 (#15, #23, #24). It also gained one commit.

  • Took the suggested wording for the oracle's array comment. json_contradiction in oracle/src/seed.rs now says any array may hold NULL elements unless nullable_items is false, which matches src/render.ts after #12. The check itself is unchanged.
  • Rebased onto #15. The PR no longer makes is_star pub(crate), because #15 already does. column_spans now sits after #15's aliased_columns. Only packages/postgres/engine.wasm was rebuilt; the MySQL and SQLite blobs are the same as at 1bd32a8.
  • Checked how the whole-row rule combines with #15. A whole-row probe is SELECT u.* …. aliased_columns gives up on a *, and hidden still covers a repeated table, so a self-joined row's probe columns stay nullable. The PR's "any probe column non-null" rule therefore cannot narrow the nulled side (whole_row_self_join keeps later: [users{…}?]). #15's new precision_rules statements have no duplicate output names and no float, numeric or record inside JSON, so neither reylo14 nor the new in-document types touch them.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@colinhacks
colinhacks force-pushed the json-soundness branch 2 times, most recently from c407a92 to e343616 Compare October 22, 2026 22:13
@colinhacks
Copy link
Copy Markdown
Owner Author

On the review of 0e78008 noting that the array[name, null] row no longer held against main after #12: right. Measured on main (f96c614), json_build_object('names', array[t.name, null]) and json_build_object('names', array[t.name]) both come back as an array with no nullable_items, so both render (string | null)[]. The description's table row and bullet now show what this PR still adds there, the narrowing to string[] where the elements are not null, and the commit message's bullet says the same.

- jsonb_agg(t.*), json_agg(t.*) filter (...), jsonb_build_object('r', t.*)
  and to_jsonb(t.*) are the whole row, not the row's first column
- an array built inside json keeps what the statement shows of its
  elements, so array[t.name] over a not null name has no null element
- a float or numeric inside json may be "NaN", "Infinity" or "-Infinity",
  and a row whose columns were not read is an object there
- two result columns of one name are an error (reylo14) at the second,
  and the row type keeps the last, as drivers do
- the oracle checks each value's kind inside a json document

Found by running SafeQL's test statements through the oracle.
@colinhacks
colinhacks merged commit 094287e into main Oct 22, 2026
6 checks passed
@colinhacks
colinhacks deleted the json-soundness branch October 22, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0