Repository navigation
fix(postgres): give a large statement the probes it needs - #50
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
<
8000
blockquote>
ℹ️ No correctness issues found. One doc comment overstates how many speculative describes get asked.
Reviewed changes
I reviewed the Postgres engine's probe budget, the one-round prewarm of pinned probes, the per-analysis cache of settled probes, the text-keyed thread-local caches and the new large_queries oracle case.
- Budget raised to 24:
MAX_PROBESgoes from 8 to 24, so large statements get the probes their JSON columns need. Past the cap, columns are left nullable rather than typed wrong (budget_spent.sql). - Casts pinned before
settle:describednow setscastsbefore the first probe, and returns pending until every parameter's oid is in the registry.insert_unknownguarantees that happens oncetypesresolves, so this cannot stall. prewarm_tree: asks for the pinned describes of FROM-item statements, set-operation arms and leaf probes in one round, recursively. It runs fromdescribedand fromrunfor the JSON shapes' probes.Engine::settled: probe outcomes are cached by text for the rest of the analysis. I traced every way a probe's outcome can depend on context: thewith_stackassumption read initem_columnsand theprobingself-meet. The globalassumptionscounter covers both, and it marks every ancestor probe too. Pending outcomes are never cached, and a cache hit only ever gives back an outcome that held in a context with no assumption, so the cache is sound.- Thread-local
stack::keptcaches: these coverparse,analyze,item_readsandlevels::needed, each a pure function of its text. Natively,stack::runruns each top-level call on a scoped thread, so these caches last for one call there. In wasm they last across rounds and edits, which is where the latency win comes from. - Oracle: there is a new
large_queriescase, andwith_probe_budget.sqlnow gets its types.tests/fingerprint.test.tspasses against the committedpackages/postgres/engine.wasm.
claude-opus-5-5 ｜ 𝕏
colinhacks
force-pushed
the
probe-budget
branch
from
October 22, 2026 23:36
a119b3f to
2045833
Compare
Contributor
There was a problem hiding this comment.
✅ No new issues found. The prewarm bound now holds per analysis, as the doc comment says.
Reviewed changes
I reviewed the one commit since the last review, which shares a single prewarm budget between the columns' probes and the JSON shapes' probes.
- Shared
Engine::prewarmedset: this one set replaces the per-callseensets indescribedand in the JSON-shape prewarm inrun. Both sites seed it with the analyzed statement, so across both sites at mostMAX_PROBESguesses are asked in each round.describedruns before the shape prewarm, so the statement tree's guesses are asked first. Any shape probes past the cap are settled in later rounds, which can cost a round but never changes a type. - Doc comment on
prewarm_tree: it now states the shared bound, which resolves the earlier thread.
claude-opus-5-5 ｜ 𝕏
A hundred-line statement needs about a dozen probes, and eight left its JSON columns unknown. The budget is 24, and the probes are cheaper: every one of them is asked in one round, a settled probe is not settled again, and the readings of a statement's text are kept.
colinhacks
force-pushed
the
probe-budget
branch
from
October 23, 2026 03:23
2fb48a9 to
8c4c2bb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A statement of about a hundred lines (three or four WITH queries, a derived table, scalar subqueries building JSON) needs about a dozen probes, and the analysis allowed eight. Past the eighth, every probe was refused, so the JSON columns read last came back with
unknownfields. Small statements with the same constructs were typed precisely, which is why only large ones showed it.oracle/cases/large_queries)mainwindow_beside_json.sql: alag()column beside ajson_aggsubquerylast_order_items: [{ sku: unknown?, … }]?[{ sku: text, name: text, quantity: int4, … }]?repeated_table_outer_join.sql: a table read again in a subquery beside aleft joinlast_order_items: [{ sku: unknown?, … }]?[{ sku: text, … }]?budget_spent.sql: a chain of fourteen WITH queries, past the new budgetn: text?where its probes stop, never a wrong typeThe budget is 24, which covers the largest of 225 variants of a hundred-line reporting query (13 probes) with room to spare. A larger budget alone made those statements slower: each probe settled the next only once described, so a statement took a round per nested probe (15 rounds for one of them, against the host's limit of 16), and each round replayed the whole analysis. So:
prewarm_tree), bounded by the budget. The casts are set before the first probe, so a probe asked early and read late is one text, one describe.stack::kept, 64 of each), which a host's replay, round after round and edit after edit, reads again.Editor latency
tsc --lspwith the content mapper, the time from an edit to a.sqlfile until the hover shows the new column (10 edits per run, 3 runs per engine, alternating, Postgres 18 on localhost; the machine was busy with other builds, so compare within a row):main→ this PRmaintop_customers.sql, 7 probes)json_over_with_param, 4 WITH queries and 2 JSON columns (13 probes)Checked
json_item_shapes/with_probe_budget.sqlgains its types; the newlarge_queriescase runs every statement over seeded values); the whole workspace builds with no warning.packages/postgres/engine.wasmrebuilt;nub run typecheckandnub run testagainst Postgres 18.