Repository navigation
fix(postgres): type a json coalesce by the default it falls back to - #10
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Important
The new coalesce fallback fixes the unsound shapes. But it also makes sibling fields in the same jsonb_build_object lose their types when they were typed correctly before. Details and a tested fix are inline.
Reviewed changes
I reviewed the whole PR: the coalesce arm in the Postgres JSON shape builder, the stricter oracle document check, the new oracle case and the docs.
coalescekeepsx's shape only when that is sound:xis kept when it is the only argument, when it is a non-null object or array, or when it is an array defaulted to'[]'(is_empty_array). Otherwise the drafts are rolled back and the wholecoalescebecomes onejsonleaf.- The oracle now checks the kind of a document: in
json_contradiction, a value typed as an object that is not a JSON object, or typed as a list that is not a JSON array, now counts as a contradiction. A unit test covers this. - New case and re-blessed output:
oracle/cases/json_coalesce_defaultsadds 8 statements.qualified_aggregatechanges fromjson?tojson. - Docs: the "Aggregates over no rows" section now says that any other default gives
unknown.
I ran the oracle against PG16 on a scratch case, built once from this PR and once from main. The PR's own cases and json_user_functions pass.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
ℹ️ The fix for the shared probe is sound. One narrower case still loses its siblings' types — details inline.
Reviewed changes
I reviewed ea3e1a3, which gives an aggregating coalesce fallback a probe of its own.
- A fallback that aggregates gets its own probe: when
crate::infer::aggregatessees an aggregate in thecoalesce, its leaf goes under a fresh context id over the sameFROMand filter. Other fallbacks stay on the context's shared probe, so they use none of theMAX_PROBESbudget. - New cases and a unit test:
sibling_aggregateandgrouped_filtercover the two statements from the earlier thread.a_coalesce_read_as_one_value_probes_an_aggregate_apartchecks both probe layouts.
I ran the reylo-engine-postgres lib tests on this head. They pass.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found. The
Shape::Unknownarm fixes the shared probe regression, and I resolved that thread.
Reviewed changes
I reviewed 787d589, which leaves a coalesce over an x the build cannot explain as Unknown.
- An unexplained
xleaves the wholecoalesceunexplained: a new(Shape::Unknown, _)arm comes before the fallback. It restoresself.draftsand returnsUnknownwithout leafing, so nothing reaches the context's shared probe. I traced every path wherewalkreturnsUnknown: overflow, a malformed constructor, aROW_FNScall with a non-alias argument, a non-EXPRsublink, andleafon a placeholder or an exhausted budget. In each one,mainalready returnedUnknown, and atea3e1a3the fallback produced either the sameUnknownor an unsound shared-probe leaf. So the arm loses no typing. - New case and unit test assertion:
unexplained_beside_columnrecords{ name: text, titles: unknown? }. The unit test checks that the probe holds onlyu.name.
I ran the jsonshape lib tests on this head. All 26 pass.
claude-opus-5-5 ｜ 𝕏
coalesce(jsonb_agg(x), '{}') over no rows is {}, yet was typed as the
aggregate's array. An array keeps its shape only under '[]' now, and
any other default makes the coalesce one value of unknown JSON. The
oracle's check of a document now fails a value of the wrong kind.
A coalesce read as one value carried its aggregate into the probe of its context, which has no GROUP BY, so a bare column beside it failed that probe and every field read from it was unknown.
787d589 to
bc1140d
Compare
There was a problem hiding this comment.
✅ No new issues found. The placeholder arm is sound, and the re-blessed
with_nested_coalescedoutput is a correct tightening.
Reviewed changes
I reviewed 6a1f908, 7f190e7 and bc1140d, which came after 787d589.
- A placeholder keeps its type under a default that cannot widen it: the new
(Shape::Param(_), defaults)arm keepsParamonly when every default is a NULL, an untyped string, an integer or a boolean constant. Anything else falls toleaf, which refuses an expression that reads a placeholder, socoalesce($1::int, 1.5)isunknown?.maintyped it asint4even though it is anumeric. Under the safe defaults the arm brings backmain's typing, which787d589had lost toUnknown. - One empty array test, shared:
jsonshapenow callscrate::infer::is_empty_array. It accepts the same nodes as the removed copy: anSval'[]', optionally cast tojsonorjsonb. with_nested_coalescedre-blessed as non-null:coalesce(ps.posts, '[]')walksps.poststo aColumn, not anArray. Since73a9521, that sends the wholecoalesceto the probe as one leaf. There,item_readssees anItemRead::Defaulted, andpostgres.rsmakes it non-null only when the item's column is aType::List. So[{ id, title }]without?is sound. Before this commit, the old expected output was stale.
I ran cargo test -p reylo-engine-postgres --lib jsonshape on bc1140d. All 27 tests pass. I did not run the oracle.
claude-opus-5-5 ｜ 𝕏
On Postgres,
coalesce(x, d)of JSON a statement builds tookx's shape wheneverdwas any non-null constant. That is wrong for an aggregate: over no rowsjsonb_aggisNULLand the value isd, soselect coalesce(jsonb_agg(id), '{}') as ids from posts where falsewas typed{ ids: number[] }and returned{}. The same held for an object from a scalar subquery that can find no row, and for a value under a non-constant default.Now
coalesce(x, …)keepsx's shape only wherexstands alone, where it is neverNULL(jsonb_build_object, or an aggregate under a plainGROUP BY), or where it is an array defaulted to'[]', the one default that is always an array. Under any other default the wholecoalesceis one value of unknown JSON, and it is non-null where a default is.select coalesce(jsonb_agg(id), '[]') as ids from posts where falseids: [int8]ids: [int8]select coalesce(jsonb_agg(id), '{}') as ids from posts where falseids: [int8]ids: jsonselect coalesce((select jsonb_build_object('id', p.id) from posts p where p.user_id = u.id limit 1), '[]') as j from users uj: { id: int8 }j: jsonselect coalesce(app.json_agg(name), '[]') as j from users(a user's aggregate)j: json?j: jsonThe oracle did not catch this because its check of a Postgres document returned no contradiction where the value was of the wrong kind: an object where a list is typed, or a list where an object is. It fails that now, as the SQLite check already did, and a unit test covers it. With the stricter check, every oracle case still passes.
Tests:
oracle/cases/json_coalesce_defaults: 8 statements, sound over the seeded edge values.oracle/cases/json_user_functions/qualified_aggregate.sql: re-blessed, non-null now.--strict-coverage, the loader's typecheck and tests, the package build, and the basic and JSON Schema labs.