8000 fix(postgres): keep an outer-joined row's own fields in to_jsonb by colinhacks · Pull Request #74 · colinhacks/reylo · GitHub
[go: up one dir, main page]

Skip to content

fix(postgres): keep an outer-joined row's own fields in to_jsonb - #74

Merged
colinhacks merged 4 commits into
mainfrom
fix/pg-to-jsonb-outer-fields
Oct 23, 2026
Merged

colinhacks merged 4 commits into
mainfrom
fix/pg-to-jsonb-outer-fields

Conversation

@colinhacks
Copy link
Copy Markdown
Owner

An outer-joined table's whole row turned into JSON (to_jsonb(p), to_json(p), row_to_json(p), p.*, and json_agg(p.*)) was typed as a nullable object whose every field was nullable too, the table's own NOT NULL columns included. The probe that types the fields reads them through the outer join, which nulls every column of an unmatched row. But an object that exists is a row that matched, so its fields hold what the row holds.

-- orders (id int primary key, user_id int not null, note text)
select u.name, to_jsonb(o) as "order" from users u left join orders o on o.user_id = u.id
  • Before: order: { id: number | null; user_id: number | null; note: string | null } | null
  • After: order: { id: number; user_id: number; note: string | null } | null

The rule

A whole row that an outer join may null is read a second time with that row always present. Each outer join above its alias is changed so it keeps the row: a LEFT JOIN with the alias on its nullable side becomes an inner join, and a FULL JOIN becomes the one-sided join that preserves the alias's side. Each field is nullable only where that read says it is. The object's own nullability doesn't change. The second read goes through the same analysis as any other statement. So a base table's field takes its NOT NULL, and a view, derived table or WITH query's field takes that item's own analysis, never attnotnull. The set of rows where the alias is present is the same under both joins, because an outer join pairs its nullable side's row only with a row whose ON condition held.

Oracle

  • json_subqueries/row_left_join: j: { id: int8?, user_id: int8?, title: text? }? → j: { id: int8, user_id: int8, title: text }?
  • merge_returning/json: a MERGE … RETURNING to_jsonb(s) source row that may be missing: row: { id: int4?, name: text?, label: text? }? → row: { id: int4, name: text, label: text }?
  • New statements in inference_precision, each checked over seeded rows:
    • left join through all four spellings
    • jsonb_agg(p.*)
    • right join and full join
    • nested join (u left join (posts p join users a …))
    • self-join told apart by alias
    • left-joined derived table, WITH query and view, each with a nullable column of its own that stays ?
    • a set-returning function (unchanged, json?)

Copilot AI balanced review requested due to automatic review settings October 23, 2026 17:30
@vercel
vercel Bot commented Oct 23, 2026 •
Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
reylo Ready Ready Preview Oct 23, 2026 5:45pm UTC

Request Review

Copilot AI left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@pullfrog pullfrog Bot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No soundness issues found. Two minor points inline: a docs sentence that claims more than the PR does, and a probe-budget ordering issue.

Reviewed changes

I reviewed the full diff: the second "row present" read for an outer-joined to_jsonb(alias), how the resolver uses it, the new oracle statements and the docs line.

  • row_present rewrite: each outer join above the alias becomes one that keeps the alias's row. LEFT and RIGHT become inner joins, and FULL becomes the one-sided join that preserves the alias. Joins that have their own alias are not entered. All three rewrites are valid SQL, since FULL's merge-joinable restriction does not apply to LEFT or RIGHT, and they keep the same set of rows where the alias is present. A USING merged column has the same value on those rows.
  • Resolver: a field is nullable only when both the main probe and the present read say it can be NULL. The object's own nullability still comes from the main probe alone, and Postgres's PlaceHolderVar keeps a pulled-up subquery's whole row NULL rather than ROW(NULL, …).
  • Engine::shapes: the present read is described over the same text the main probe used (narrow or wider) and is checked to have the same number of columns.
  • Oracle and docs: two blessed cases have tighter fields, there are ten new whole_row_json_* statements, and the "Whole rows" sentence in postgres.mdx changed. The committed engine.wasm matches its sources (tests/fingerprint.test.ts passes at c54afea).

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread site/content/docs/postgres.mdx Resolved
Comment thread engine/postgres/src/postgres.rs Resolved
@pullfrog pullfrog Bot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. Both earlier points are fixed.

Reviewed changes

I reviewed the two commits pushed since the last review, 9d77016 and e465783.

  • Probe order in Engine::shapes: the function now describes every main probe (narrow, then wider) first, and only then each present read, so a present read can no longer use up budget that a later main probe needs. Each present read still uses the same text as its main probe: narrow if the narrow probe succeeded, otherwise wider. A probe that is still pending is skipped. Present reads still run before items are followed, just as they did before this change.
  • Docs: the "Whole rows" sentence in postgres.mdx now limits kept field nullability to rows turned into JSON. It no longer claims this for composite array_agg(p).
  • Artifact: the committed packages/postgres/engine.wasm matches its sources (tests/fingerprint.test.ts passes at e465783).

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@colinhacks
colinhacks merged commit e71455b into main Oct 23, 2026
6 checks passed
@colinhacks
colinhacks deleted the fix/pg-to-jsonb-outer-fields branch October 23, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0