Repository navigation
feat(postgres): read each column's type modifier - #180
Conversation
|
The latest updates on your projects. Learn more about Vercel for Gi 8000 tHub.
|
There was a problem hiding this comment.
Important
The schema fingerprint that keys .reylo/ doesn't track type modifiers, so a migration that changes only a modifier keeps serving the old descriptor. There is also one oracle false positive, noted inline.
Reviewed changes
I reviewed the full diff of e41b588: the hosts now send RowDescription's typmod, the Postgres engine decodes it into the type descriptor, and the oracle's value checks are stricter.
- Hosts send
typmod: the native program, the facade's tokio-postgres client and the PGlite host each forward each column's type modifier. A missing one defaults to-1. - Type registry reads modifiers: the type query now also reads
typtypmodand each composite field'satttypmod.render_atpasses a modifier through arrays, domains and composite fields, andwith_typmoddecodes the Postgres and PostGIS encodings. I checked these againstvarchar,numeric(the 11-bit signed scale), interval masks andTYPMOD_GET_*, and they match. - Result columns use it: the plain-column path in
postgres.rsnow callsrender_typmod. Parameters of a domain or composite type pick up the catalog's modifiers throughrender. That matches the spec's rule that a parameter may be stricter than the server, never looser. - Oracle is stricter:
seed::contradiction, which the Postgres and SQLite runs use, now checks text lengths, decimal precision and scale, ticks, interval parts, and bit and vector lengths. There is a newtype_modifierscase, and three blessedexpected.txtfiles changed.
⚠️ The schema fingerprint ignores type modifiers, so cached analyses go stale
A Postgres analysis now depends on atttypmod and typtypmod. But POSTGRES_FINGERPRINT in src/analyze.ts hashes a.atttypid and t.typbasetype and leaves out both modifiers. So alter table t alter column name type varchar(50) on an unindexed column leaves the fingerprint unchanged, and the .reylo/ entry keeps varchar(≤20). Changing numeric to numeric(10,2) has the same problem: the cached output JSON Schema keeps or drops Infinity according to the old type.
Technical details
# Fingerprint the type modifiers the analysis now reads
## Affected sites
- src/analyze.ts:560 — the `a` line of `POSTGRES_FINGERPRINT` hashes `atttypid` but not `atttypmod`
- src/analyze.ts:568 — the `t` line hashes `typbasetype` but not `typtypmod` (`create domain d as varchar(n)`)
- src/analyze.ts:525-536 — the doc comment listing what the fingerprint covers
## Required outcome
- A change to a column's or a domain's type modifier, with nothing else changed, changes the Postgres fingerprint, so the `.reylo/` entries keyed on it are analyzed again.
## Suggested approach
- Add `out("int4", "a.atttypmod")` to the `a` line and `out("int4", "t.typtypmod")` to the `t` line, keeping the `textin(int4out(…))` form the query uses for search-path safety.
- Add a test beside the existing fingerprint tests: alter a column's type modifier and assert that the fingerprint moves.ℹ️ Fields built into JSON don't get the modifier
ResolveCtx::column in engine/postgres/src/jsonshape.rs:1481 still calls registry.render(col.type_oid). So json_build_object('total', total) types total as plain numeric (check_non_finite/built.sql), while to_jsonb(row) gives numeric(10,2) through the composite's fields (whole_rows.sql). This is sound, just less precise. If the omission isn't intended, passing col.typmod through render_typmod there would make the two paths agree.
claude-opus-5-5 ｜ 𝕏
|
Both points from the review are addressed:
|
There was a problem hiding this comment.
✅ No new issues found. The three commits address each point from the prior review.
Reviewed changes
I reviewed the three commits since e41b588: a fix to the fingerprint, a fix to the oracle's decimal check, and type modifiers in fields built into JSON.
- Fingerprinted type modifiers:
POSTGRES_FINGERPRINTnow hashesa.atttypmodandt.typtypmod. The fingerprint test asserts that changingvarchar(20)tovarchar(50)moves it, and so does changingnumerictonumeric(10,2). - Fixed the decimal precision count:
decimal_contradictionnow counts the significant digits of the whole and fractional parts together, with leading zeros trimmed. So0.000123fits anumeric(3,6), while0.001230breaks it. A new unit test covers that case, negative scales and ordinary values. Zero under a negative scale still passes, because the trimmedwholeis empty. - Kept modifiers in JSON-built fields:
ResolveCtx::column8000 now callsrender_typmod. As a result,json_build_object('total', total)givesnumeric(10,2), which matchesto_jsonb(row). Thecheck_non_finiteandpgvector_typesexpected files change to match.
claude-opus-5-5 ｜ 𝕏
The Postgres engine reads each column's type modifier, so a column's type descriptor carries the length, precision or tick it was declared with:
varchar(255)is text of at most 255 code points,numeric(10,2)a decimal of precision 10 and scale 2,timestamp(3)an instant ticking by a millisecond.varchar(20){ "kind": "text", "size": { "max": 20 }, "name": "varchar" }char(5){ "kind": "text", "size": { "exact": 5 }, "name": "bpchar" }, since it pads what it storesnumeric(10,2){ "kind": "decimal", "precision": 10, "min_scale": 2, "max_scale": 2, "nan": true, "name": "numeric" }: NaN, and no infinity, which a sizednumericrefusesnumeric(5,-2)time(3),timestamp(3),timestamptz(0)tick: "1/1000", or"1", and the latest value cut to the tickinterval year to month,interval day,interval hour,interval hour to minute,interval second(3)bit(7),varbit(5)vector(3),halfvec(3)varchar(20)[]geometry(Point, 4326),geometry(LineStringZ, 3857)srid,subtypeandcoordinates:4326,point,xyWhere the modifier comes from
typmod, -1 for none): the native program, the facade's tokio-postgres client and the docs site's PGlite host. It covers a table's column, a view's, a cast (name::varchar(3)) and an expression that keeps one (coalesce(name, name),tags[1]). A column of a domain is described as its base type under the domain's modifier.pg_type.typtypmodandpg_attribute.atttypmod, read with the type itself, socreate domain email as varchar(255)and a composite'szip varchar(10)carry their sizes inside rows and JSON too.A host that sends no
typmodgives the engine none, which leaves every type as it was. Parameters are unchanged: the describe reports no modifier for one.What changes for a user
The declarations and the values
.run()returns are the same: the docs capture is unchanged, all 182 modules.reylo analyzeprints the new fields, and a sizednumericcolumn's JSON Schema leaves outInfinityand-Infinity, which it cannot hold.Checks
type_modifiers, reads each modifier through a column, a cast, a domain, a composite, an array, a view and an interval's fields, over rows at each type's edges. The oracle now holds a value to its size, its precision and scale, its tick and its interval parts, besides what it held before; Postgres's and SQLite's 127 cases pass under the stricter checks.expected.txtfiles change, each a column gaining its modifier:check_unions(varchar(≤10, 'free' | 'pro'),bpchar(=3)),check_non_finite(numeric(10,2), a composite's field included) andpgvector_types(vector(3),halfvec(3)).cargo build --workspace --all-targetswith no warning, every crate's tests,nub run typecheck, andnub run testwith both test databases (340 passed, none skipped).