Repository navigation
feat(sqlite): compile each statement on cloudflare d1 through wrangler - #44
Conversation
D1 refuses functions its SQLite has, such as json_pretty and power, through an allowlist nothing lists, so a statement is compiled with EXPLAIN on D1 as the project's wrangler runs it locally.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Caution
With a D1 binding marked "remote": true, this PR makes the type check run the local schema's DDL on the real remote D1 database. The schema D1 sees is also a snapshot that never refreshes in the editor, so after a migration it reports false errors.
Reviewed changes
I reviewed the D1 check end to end: the proxy child, how the driver opens and its IPC, the refusal wording and placement, the tests and the docs.
src/d1-proxy.ts: a forked child calls wrangler'sgetPlatformProxy({ persist: false }), replays the schema's DDL into the first D1 binding and answersexplain <sql>requests over IPC, binding onenullper parameter.openD1insrc/sqlite-drivers.ts: reads the DDL throughnode:sqlite, forks the proxy and waits up to 30s for it to start. The IPC channel holds the event loop only while a request is pending.SqliteDriverinterface:versionis replaced bysays(refusal), so each driver words its own message.refusesnow takes the binding count (rewritten.order.length).placeRefusal: also places D1'snot authorized to use function: xat the call.- Tests and docs:
wranglerjoinslab/drivers. New tests cover D1's refusals ofjson_prettyandceiling, statements it allows, and a config with no D1 binding. The SQLite reference page and the example README describe the check.
claude-opus-5-5 ｜ 𝕏
A binding marked remote is never proxied to the real database, D1 gets the schema again whenever it changes, and workerd starts once per project rather than with each connection.
There was a problem hiding this comment.
ℹ️ All four earlier findings are fixed. One minor suggestion inline, about the failure path of the new per-project cache.
Reviewed changes
I reviewed facbc5f, which keeps D1 local, keeps its schema current, and runs one child per project.
- Local only:
getPlatformProxynow getsremoteBindings: false, so a binding markedremotenever reaches the production database. - Schema kept current:
refusesreads the DDL again for each statement, and the child gets a newschemaonly when the DDL differs from the last one sent.replay()drops the earlier views and tables before it runs the new DDL. The child handles one message at a time, so a new schema is in place before the statements sent after it compile. - One child per project: the new
d1smap, keyed by wrangler entry and project directory, holds the D1 child for the life of the process. Closing a connection now closes only its read handle, andended()removes a child from the map when it exits. - Test: a session adds a column and a table partway through, then calls
recheck()and expects no diagnostic. A later session reuses the child.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found. The earlier finding about failed D1 starts is fixed, and I resolved its thread.
Reviewed changes
I reviewed 78d1a69, which remembers a failed D1 start until the wrangler config changes.
- Failed starts stay cached: a
d1sentry is now{ config, started }, and a rejectedstartedstays in the map. A project that binds no D1 database, or whose workerd times out, starts workerd once per process instead of once per connection. - Config-keyed invalidation:
wranglerConfig(dir)records the name, size and mtime ofwrangler.json,wrangler.jsoncandwrangler.toml. When the record no longer matches, the old child is closed through the newD1Process.closeand a new one starts. - Eviction only after a successful start: a
runningflag meansended()runs only for a child that exits after it started. Thed1s.get(key)?.started === startedguard stops a closed child's exit from evicting the child that replaced it. - Test: the no-binding test now runs two sessions, then rewrites the config with a D1 binding and expects the
json_prettyrefusal.
claude-opus-5-5 ｜ 𝕏
For a project whose
package.jsonlistswranglerand whose wrangler config binds a D1 database, the type check now compiles each SQLite statement on Cloudflare D1 as that wrangler runs it locally. A statement D1 refuses is areylo6error at the call:Why. D1 refuses functions its SQLite has, through an authorizer that runs while SQLite compiles a statement, and it refuses
pragma_function_listtoo, so nothing can list what it allows. Probing every functionnode:sqlitehas through local D1, withEXPLAIN, found 127 allowed (all on the allowlist in Cloudflare's runtime source,ALLOWED_SQLITE_FUNCTIONSin workerd'ssrc/workerd/util/sqlite.c++), 22 refused, among themjson_pretty,unistrand the aliasesceiling,log10andpowerwhereceil,logandpoware allowed, and 21 absent from D1's build (FTS3 and FTS4, Geopoly,percentile). All of these passed the type check, which reads the local D1 file throughnode:sqlite, and failed in the Worker.How.
src/d1-proxy.tsruns in a child process, because wrangler and workerd write to stdout, the content mapper's channel to TypeScript. It calls wrangler'sgetPlatformProxy({ persist: false }), takes the config's first D1 binding (in memory, so nothing is written under.wrangler/), replays the schema of the database the type check reads into it, and answersEXPLAIN <statement>requests over IPC, binding one null per parameter.src/sqlite-drivers.tsadds it as one more driver beside those of the previous change, and each driver now words its own refusal.Checks
tests/sqlite-drivers.test.ts: D1's refusals ofjson_prettyandceilingwith their message and position;ceil, aninsert … returningand a two-argumentiifwith their parameters bound and no error; a project whose config binds no D1 database.lab/driversgainswrangler, which the tests load; CI installs it.examples/cloudflare-workers, whose config binds D1, type-checks, tests, serves, builds and dry-runs a deploy with the check on.