8000 perf: prepare postgres.js statements so a run takes one round trip by colinhacks · Pull Request #107 · colinhacks/reylo · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions site/content/docs/postgres.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -830,7 +830,7 @@ An error inside a transaction aborts it, even when `.try()` returns the error: e

## Drivers

Reylo decodes its query results itself, so type parsers and options you set on your client don't affect Reylo's queries, and Reylo's don't affect yours. Type checking reads your schema through the postgres.js that `@reylosql/postgres` carries, whatever driver your app uses.
Reylo decodes its query results itself, so type parsers and transforms you set on your client don't affect Reylo's queries, and Reylo's don't affect yours. Type checking reads your schema through the postgres.js that `@reylosql/postgres` carries, whatever driver your app uses.

Each example below runs this query:

Expand Down Expand Up @@ -879,7 +879,7 @@ await getUser.run(reserved, { id: 1 });
reserved.release();
```

A client created with `transform.row` isn't supported.
Reylo prepares its queries on each connection, as postgres.js prepares its own. Behind a pooler in transaction mode, such as PgBouncer's, create the client with `prepare: false`, and Reylo's queries run unprepared too. A client created with `transform.row` isn't supported.

### `pg-promise`
8000
Expand Down
2 changes: 1 addition & 1 deletion skills/using-reylo/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The docs are one page, https://reylo.dev, also served as Markdown at https://rey

## Running a query

- Pass the project's own database client to `.run()`: a pool, a client, a transaction or connection taken from one, or a Drizzle database over one (https://reylo.dev/#supported-drivers, https://reylo.dev/#transactions). At run time Reylo opens no connection of its own.
- Pass the project's own database client to `.run()`: a pool, a client, a transaction or connection taken from one, or a Drizzle database over one (https://reylo.dev/#supported-drivers, https://reylo.dev/#transactions). At run time Reylo opens no connection of its own. On postgres.js, Reylo prepares each query as postgres.js prepares its own; behind a pooler in transaction mode, such as PgBouncer's, create the client with `prepare: false`, which Reylo's queries follow too.
- `.runSync()` returns the result itself on a synchronous SQLite client (`better-sqlite3`, `node:sqlite`, a Durable Object's `ctx.storage.sql`), so it works inside their synchronous transactions. `.with(client)` binds the query to a client.
- The result's shape follows the row count Reylo infers: an array, one row, a row or `null`, or nothing for a write without `returning`. On Postgres and SQLite, a subquery, a `with` query or a view that selects a table's key unchanged is matched on that key like the table.
- Every driver gives the same types: a 64-bit integer is a `bigint`, a decimal its exact text, an instant its ISO 8601 text in UTC, a pgvector `vector` a `number[]`, an `hstore` an object of its keys, and no value is a `Date`. Pass a parameter in the form `.run()` returns for its type (https://reylo.dev/#data-types).
Expand Down
6 changes: 5 additions & 1 deletion src/decode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,11 @@ type BufferLike = Uint8Array;
const textOf = utf8;

export async function postgresJs(c: AnyClient, text: string, params: unknown[], columns: Columns | undefined): Promise<unknown[]> {
const pending = c.unsafe(text, params);
// Prepared, as postgres.js prepares its own queries: its unsafe() prepares nothing by default,
// so every run with a parameter took two round trips (Parse and Describe, then Bind and Execute)
// where a statement prepared on the connection takes one. A client made with `prepare: false`,
// as one behind a pooler in transaction mode is, still prepares nothing: its option wins.
const pending = c.unsafe(text, params, { prepare: true });
// A client whose query has no raw() (a wrapper of postgres.js's shape): its values, given their kinds.
if (typeof pending?.raw !== "function") return bunRows(await pending, columns);
const raw = await pending.raw();
Expand Down
55 changes: 53 additions & 2 deletions tests/execute.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { existsSync } from "node:fs";
import { createRequire } from "node:module";
import { join } from "node:path";
import { DatabaseSync } from "node:sqlite";
import { fileURLToPath } from "node:url";
import { arrayLiteral, execute, executeSync, type QueryModule } from "../src/execute.ts";
import { checks, defineQuery } from "../src/index.ts";
import type { EngineType, JsonSchema } from "../src/engine-type.ts";
Expand Down Expand Up @@ -41,11 +45,16 @@ test("postgres: pg-promise gets a { text, values } object, so it binds instead o
assert.equal(types.getTypeParser(20)("9007199254740993"), 9007199254740993n);
});

test("postgres: unsafe(text, values) → rows, as on postgres.js and Bun.sql and their transactions", async () => {
test("postgres: unsafe(text, values) → rows, as on postgres.js and Bun.sql and their transactions; postgres.js is asked to prepare the statement", async () => {
const calls: Call[] = [];
const tx = { unsafe: (...args: unknown[]) => (calls.push({ method: "unsafe", args }), Promise.resolve([row])) };
assert.deepEqual(await execute(tx, pg, { id: 1, name: "ada" }), [row]);
assert.deepEqual(calls, [{ method: "unsafe", args: ["select $1::int as id, $2 as name, $1 as again", [1, "ada"]] }]);
assert.deepEqual(calls, [{ method: "unsafe", args: ["select $1::int as id, $2 as name, $1 as again", [1, "ada"], { prepare: true }] }]);
// Bun.sql prepares a statement itself, and takes no options.
const bunCalls: Call[] = [];
const bun = { unsafe: (...args: unknown[]) => (bunCalls.push({ method: "unsafe", args }), Promise.resolve([row])), commitDistributed: async () => {} };
assert.deepEqual(await execute(bun, pg, { id: 1, name: "ada" }), [row]);
assert.deepEqual(bunCalls, [{ method: "unsafe", args: ["select $1::int as id, $2 as name, $1 as again", [1, "ada"]] }]);
});

// The module's `inputs`: :ids is an int4[] parameter, :data a jsonb one, :name text.
Expand Down Expand Up @@ -368,6 +377,48 @@ test("postgres: Neon's neon() before 1.0 is called with (text, values)", async (
assert.deepEqual(Object.keys(calls[0]!.args[2] as object), ["fullResults", "arrayMode", "types"]);
});

// postgres.js on a throwaway server (REYLO_TEST_PG), in a database of its own; the driver is lab/drivers'.
const PG = process.env.REYLO_TEST_PG;
const DRIVERS = fileURLToPath(new URL("../lab/drivers/node_modules", import.meta.url));
8000 const pgLive = !PG ? "REYLO_TEST_PG names no server" : !existsSync(join(DRIVERS, "postgres")) ? "lab/drivers is not installed" : false;

test("postgres.js: a statement with parameters is prepared on its connection and run again from there, unless the client was made with prepare: false", { skip: pgLive }, async () => {
const postgres = createRequire(join(DRIVERS, "postgres", "package.json"))("postgres");
const admin = postgres(PG, { max: 1, onnotice: () => {} });
const name = `reylo_execute_prepare_${process.pid}`;
await admin.unsafe(`drop database if exists ${name}`);
await admin.unsafe(`create database ${name}`);
const url = new URL(PG!);
url.pathname = `/${name}`;
const prepared = postgres(url.href, { max: 1, onnotice: () => {} });
const unprepared = postgres(url.href, { max: 1, prepare: false, onnotice: () => {} });
const q: QueryModule<"postgres"> = { dialect: "postgres", sql: "select id, name from users where id = :id::int", params: ["id"] };
const text = "select id, name from users where id = $1::int";
// pg_prepared_statements lists the statements of the connection that reads it, and how often
// each ran: two plans for two runs is one statement prepared once and run twice.
const statements = async (c: { unsafe(text: string): Promise<Record<string, unknown>[]> }) =>
[...(await c.unsafe("select statement, custom_plans + generic_plans as runs from pg_prepared_statements where statement = '" + text + "'"))].map((r) => ({ ...r }));
try {
await prepared.unsafe("create table users (id int primary key, name text not null); insert into users values (1, 'Ada')");
for (const client of [prepared, unprepared]) {
const reserved = await client.reserve();
try {
for (let i = 0; i < 2; i++) assert.deepEqual((await execute(reserved, q, { id: 1 })).map((r) => ({ ...(r as object) })), [{ id: 1, name: "Ada" }]);
assert.deepEqual(await statements(reserved), client === prepared ? [{ statement: text, runs: "2" }] : []);
} finally {
reserved.release();
}
}
// A transaction runs on one connection of the client, prepared too.
assert.deepEqual(await prepared.begin(async (tx: never) => (await execute(tx, q, { id: 1 }), statements(tx))), [{ statement: text, runs: "3" }]);
} finally {
await prepared.end();
await unprepared.end();
await admin.unsafe(`drop database if exists ${name}`);
await admin.end();
}
});

test("a parameter left out binds NULL, which postgres.js, mysql2, node:sqlite and libSQL would reject", async () => {
const seen: unknown[][] = [];
const client = { unsafe: async (_: string, values?: unknown[]) => (seen.push(values!), []) };
Expand Down
Loading
0