Repository navigation
feat(cli): add reylo migrate - #179
colinhacks wants to merge 40 commits into
Conversation
Migrations are checked against schema.sql in shadow databases before they run, and create writes the diff engine's plan, verified in a shadow, in schema.sql's style. history: linear keeps migrations/lock.yaml, which rebase resolves from git's index.
apply never records a migration without running it unless --record names one with no data statements, which shadows prove the database already has; an applied migration edited since is refused with both texts, and restore writes back the one that ran. check holds the migrations against schema.sql and runs nothing. Postgres plans convert columns by assignment, and read a table's access method. A SQLite database is locked for a run through a file in .reylo/.
create writes a migration only when --drop names every table and column it drops with their rows, and every name --drop gives is one it drops; --rename states a column's rename, which may change its type. The planner decides renames once, for every database's alterer: a column --drop names is never guessed renamed. Drafts not yet applied anywhere no longer stop create.
sync takes a development database to schema.sql through the migrations: its history is kept as far as it agrees with the files, the database is rewound past that, the migrations after it run for real, and what schema.sql has beyond them is made unrecorded. A migration with no data statements whose change the database already has is recorded instead. It refuses when CI is set. wipe deletes every row of the project's tables and runs the seed.
sync keeps what an earlier sync made unless a pending migration cannot run on it, and checks each plan it runs before it commits. create counts a replaced column or table as a drop, resolves --drop and --rename names exactly, and repeats them in its suggestions. wipe leaves SQLite's virtual tables' own tables alone. apply drops an index a cut-off concurrent build left invalid, and --record takes the name create was given.
A table that goes while one of its columns comes is renamed, or as --rename states, with what Postgres names after it (keys, sequences, partitions' indexes). sync's rewind undoes only what the rewound migrations made, renaming tables back, and keeps what it records.
Every migration command is under reylo migrate. apply --dev takes a development database to its migrations; sync, which refuses until it is there, makes what schema.sql adds. sync-history replaces --record and repair: it records the longest run of migrations the database's schema already has, proven in a shadow. rebase is gone.
A project is locked unless history is loose. The lock lists each migration with a checksum chained to the one before, and schema.sql's checksum when the migrations reach it. apply runs only what it lists, in its order; a file it lacks is a draft, which apply --dev runs and locks. A conflicted lock is resolved by keeping main's lines and running apply --dev. lock writes it from a database's history, unlock removes it, and check --offline checks it without a database.
The project's files and caches live in the directory the top-level dir names, reylo/ by default. apply replays nothing where nothing is pending and the database is as its last run left it, and trusts a run's result only where it matches the replay. create gates type changes that lose values with --drop, and assumes renames where one table or column goes and one like it comes. Partitions made at run time are data. sync --watch syncs again on each change.
The cache moves from .reylo/ to the .cache/ of the directory dir names, beside the migrations, and keeps itself out of git there.
A page for reylo migrate, the dir and migrate keys under Configuration, the cache's new place, and the skill's Migrations.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
ℹ️ No blocking issues in the parts reviewed here, but parts of the migrate engine were not reviewed — see the coverage note below.
Reviewed changes
This run read the CLI, the project file, CI, the loader's cache move, lock.rs, lib.rs and the docs closely. Specialist passes on the rest of the migrate engine were started but stopped before they reported.
reylo migrateandreylo run: new subcommands inengine/cli/src/ma 8000 in.rs, with an argument parser of their own and an in-place edit ofmigrate.historyin.reylo.yaml(with_history).- New crate
reylo-migrate: the commands, the lock with chained checksums, shadow-database replays and their cache, a schema comparator, and diff engines for Postgres, MySQL and SQLite. - Project file: a new top-level
dirkey and amigratesection (url,shadow_url,history,strict_comments,post) inengine/core/src/config.rs. - Loader cache move: the cache moves from
.reylo/to<dir>/.cache/throughResolvedProject.cache; nothing reads an old.reylo/. - CI: the engine job gets Postgres 18 and MySQL 8.4 services and
-p reylo-migrate, and the scope filter treatsengine/migrate/*as part of the binary. - Docs: a new
migrations.mdx, thedirandmigratekeys in04-configuration.mdx, the FAQ and the skill.
ℹ️ Parts of the migrate engine were not reviewed
This is not an approval of the migration engine itself. Five questions went to specialists and none of them reported back: whether each diff engine (diff/postgres.rs, diff/mysql.rs, diff/sqlite.rs) can plan data loss that --drop does not catch, the rename heuristic among them; whether strict apply keeps its promises (commands.rs); the statement splitter (sql.rs); and whether the replay cache (cache.rs) can ever change an outcome. commands.rs (2986 lines) and diff/postgres.rs (1965 lines) are too large for the review's diff file and were only spot-checked. Those paths still need a reviewer.
ℹ️ binaries.yml doesn't run on changes to engine/migrate/
The reylo binary now links reylo-migrate, which brings in its own rusqlite, mysql_async and tokio-postgres. .github/workflows/binaries.yml still triggers only on engine/cli/** and engine/native/**, so a later change to engine/migrate/ alone never builds the binary on the eight release platforms before a release does.
Technical details
# binaries.yml path filter misses engine/migrate
## Affected sites
- .github/workflows/binaries.yml:17-18 — `paths` lists `engine/cli/**` and `engine/native/**` only
- AGENTS.md, "binaries" bullet under Checks — names the binary's program as `engine/cli/`, `engine/native/`
## Required outcome
- A pull request that changes only `engine/migrate/**` runs `binaries.yml`, and AGENTS.md says so.ℹ️ Some .reylo/ mentions are still there
Four places outside the diff still name the old cache directory: lab/bundlers/.reylo.yaml:1, lab/bundlers/README.md:11, lab/jest/.reylo.yaml:1 and the comment at engine/native/src/sqlite.rs:81.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
Important
The new CI step installs pg_dump 18, but on the runner image pg_dump still runs version 16, so the migrations' init tests will still fail.
Reviewed changes
Since the last review, one commit changes the engine job: it now installs a newer pg_dump before cargo test.
- Install
pg_dump18: adds pgdg'spostgresql-client-18to the engine job in.github/workflows/test.yml, so thatreylo_migrate::init(engine/migrate/tests/scenarios.rs:479,:1442) can dump a Postgres 18 server.
claude-opus-5-5 ｜ 𝕏
|
Answering the review's notes:
|
There was a problem hiding this comment.
Important
The new commits fix what they set out to fix. One earlier thread is still open: the pg_dump 18 step at .github/workflows/test.yml:271. CI on 5a7551b failed in the way that thread predicted: init_writes_and_records_the_baseline::postgres and init_locks_the_baseline_in_a_locked_project::postgres failed with "init needs pg_dump 18 or newer". None of these commits change test.yml or dump_from's candidates, so the engine job will fail again. There is also one minor suggestion inline.
Reviewed changes
This run reviewed the four commits since 5a7551b.
- Kept a lock's text after a backslash-escaped quote:
canonicalnow reads each'…'and"…"string both ways, with and without backslash escapes. Where the two readings end the string in different places, it keeps everything after the string as written. The new tests cover MySQL's\', Postgres'sE'…'and a double-quoted MySQL string. - Keyed sessions by their cache directory:
project.cacheis now part of bothSession.for's key and the mapper'sconfigIdentity, so a change todirreaches a running process. - Built the binaries on
engine/migrate/changes: addedengine/migrate/**tobinaries.yml's paths, and updated AGENTS.md to match. - Renamed the old cache in the remaining places: replaced the leftover
.reylo/mentions inlab/bundlers,lab/jestandengine/native/src/sqlite.rswithreylo/.cache/.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
ℹ️ No new issues. The
pg_dump18 fix works as intended. One earlier minor thread is still open, oncanonicalatengine/migrate/src/lock.rs:88.
Reviewed changes
This run reviewed the one commit since 991d15a.
- Put
pg_dump18 first on the engine job's path: the step now adds/usr/lib/postgresql/18/binto$GITHUB_PATHand checks that binary's version.dump_fromrunspg_dumpby name, socargo testnow finds 18 beforepg_wrapper, which would pick the runner's version 16. This resolves the earliertest.yml:271thread. CI on this commit was still running when this review was written.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
ℹ️ No blocking issues. Both commits do what they set out to do, and the earlier
canonicalthread is resolved. One minor gap of the same kind is noted inline.
Reviewed changes
This run reviewed the two commits since 507d64e.
- Kept text in a lock that MySQL runs:
canonicalnow keeps/*! … */as written. It also keeps the rest of the text as written after a--followed by anything other than a space or control character, and after a block comment that Postgres's nesting and MySQL's or SQLite's first*/end at different places. New tests cover each case. - Looked for
pg_dumpin Debian's versioned directory:dump_fromnow tries/usr/lib/postgresql/{major}/bin/pg_dumpafter Homebrew's keg and beforePATH'spg_dump. This means a Debian or Ubuntu machine whosepg_wrapperpicks an older local cluster still finds a matchingpg_dump.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
ℹ️ No new issues in this commit, and the earlier
#thread oncanonicalis resolved. This is not an approval, because the migrate engine as a whole still hasn't been reviewed (see below).
Reviewed changes
This run reviewed the one commit since 7866adc.
- Kept a lock's text after a
#or an unended comment: outside quoted text,canonicalnow keeps everything after a#as written. MySQL reads#as a comment and Postgres reads it as an operator. It also keeps everything after a/*that has no*/as written, where it used to drop it. I compiled the newcanonicalon its own and checked the cases in the new tests, plus'#'inside a string, Postgres's#>>,/*/ … */and/*! … */. Each one gave the expected output.
ℹ️ The migrate engine still hasn't been reviewed in depth
The first review's note still applies. No reviewer has yet checked these parts of the migrate engine:
- whether the diff engines (
diff/postgres.rs,diff/mysql.rs,diff/sqlite.rs) can plan data loss that--dropdoesn't catch, including the rename heuristic; - whether strict
applyincommands.rskeeps its promises; - the statement splitter in
sql.rs; - whether the replay cache in
cache.rscan ever change an outcome.
Every incremental run since then has reviewed only its own delta.
claude-opus-5-5 ｜ 𝕏
prod builds no shadow database: it trusts the fingerprint its last run stored, and refuses a database that no longer has it. check is what replays, and dev and reset are development's. The shadow flags, restore and migrate.post are gone. A rename create guesses that changes more than a name is refused, with one command for each way of settling them all. Runs print one aligned row per migration.
The page leads with schema.sql, shows each change as a diff, gives every migrate key with its default, and documents the CLI as help text with one heading per command. The help fits one line per command, and suggested commands print several flags one to a line.
A flag's value after an = is its value, and what a command doesn't take is refused. A # in a password stays in the credentials, for shadow databases and headings alike. reset refuses before it drops. dev stores the fingerprint of the replay, and a failure's leftovers are seen by the next prod. The fingerprint leaves out only a child's own lines. Commands are quoted for the shell, and a child's column follows its parent's rename. MySQL keeps its reserved words quoted.
There was a problem hiding this comment.
Important
The split into prod and dev reads well, and the fixes in 740acc9 hold up. One path in dev drops data the user never asked to drop: a draft migration that is simply broken makes dev undo what sync made, rows and all, before it fails anyway. There are two smaller points below. They are in the body because GitHub can't anchor comments in engine/migrate/src/commands.
10BC8
rs, which is too large for its diff view.
Reviewed changes
This run reviewed the three commits since 75c83fb. It skipped what the two chore: merge main commits brought in.
- Split
applyintoprodanddev:prodbuilds no shadow database and trusts the fingerprint it stored.dev,resetandwipeare separate commands now, each refused underCI.apply,--shadows/--no-shadows,restoreandmigrate.postare gone. - Refused flags a command doesn't take: the CLI checks the command name before parsing. It refuses any switch,
--drop/--rename/--schema/--fileor extra word that the command doesn't take, and it accepts--flag=value. - Made
prod's fingerprint survive a partial failure: where the next migration could leave part of itself (on MySQL, or a statement that runs alone), the fingerprint is read before it runs. The fingerprint also now records whether comments were compared. - Changed how
devandresetrecord their state:devnow stores the fingerprint of a shadow replay of the migrations, not of the dev database.resetchecks the lock against the history from before the wipe, then runs throughdev_on. - Rewrote
catalog::of_table: it now reads where each kind of line names its table, so a view or function that only mentions a table no longer counts as part of it. - Diff engine: child tables' columns follow a parent column's rename without a statement of their own, and quoted column names are kept as written.
- URLs:
with_database,database_ofandshown_urlnow handle a password that holds#or@. - Docs:
migrations.mdxis rewritten asmigrate.mdx(/migrate), with the CLI help, the skill and AGENTS.md updated to match.
⚠️ dev drops what sync made when a draft migration is broken on its own
At commands.rs:2757-2796, runs is false whenever a pending migration fails on the probe shadow, including when the migration is just broken (a typo in a backfill). dev then runs take_to anyway, which drops every change no migration made, with its rows. The real run at 2806-2811 then fails on the same error. One typo in a draft costs the user their sync-made tables and the data in them, and nothing is gained.
Technical details
# dev undoes unrecorded changes even when the pending migration is broken on its own
## Affected sites
- engine/migrate/src/commands.rs:2757-2775 — the probe sets `runs = false` for any failure of `run(shadows.db(lane), m, false)`. That includes a migration that fails on any schema, not only one that conflicts with the extra changes.
- engine/migrate/src/commands.rs:2776-2796 — `take_to(db, …, &expect, …)` then drops everything outside `kept + records`, and `dropped_row` reports it after the fact.
- engine/migrate/src/commands.rs:2806-2811 — the real `run` of the same migration then fails, so the run ends with exit 1 and the data already gone.
## Required outcome
- `dev` undoes changes no migration made only when the pending migrations would run without them. Where they fail on a clean replay of `kept + records` too, it reports the migration's own error and changes nothing.
- A scenario test: a `sync`-made table holding rows, plus a draft whose data statement fails. `dev` exits 1, and the table and its rows are still there.
## Suggested approach (optional)
- Before the `take_to`, replay `kept + records + pending` on a fresh lane (`shadows.lines`). If that fails, `return fail("<migration> fails when replayed in a shadow database", e, log)` before touching the target.ℹ️ prod's drift guard lapses whenever the stored fingerprint can't be compared
When there is no fingerprint to compare, prod (commands.rs:1097-1111) only logs a dim note. It then runs whatever is pending, and stores a fresh fingerprint over whatever drift exists. The tests make clear this is intended (prod_goes_on_where_it_has_no_fingerprint_to_compare). But it applies after any minor server upgrade, because server_version_num is part of the prefix how compares. It also applies after a change to catalog.rs, and after a run cut off between two migrations: insert_row writes fingerprint as NULL, and set_fingerprint only runs after the loop. migrate.mdx:310 says only that prod "refuses a database that no longer has it".
Technical details
# prod's drift guard lapses whenever the stored fingerprint can't be compared
## Affected sites
- engine/migrate/src/commands.rs:1097-1111 — `unchanged == None` logs, then runs or re-stores without any check
- engine/migrate/src/commands.rs:1150-1157 — the fingerprint is stored once, after every pending migration. A process killed after migration 1 of 3 leaves the last applied row with `fingerprint = NULL`, so the next run is unchecked.
- engine/migrate/src/commands.rs:761-762 — `how` compares `code_id:server_version:comments`, so a minor upgrade on a managed host (18.1 → 18.2) makes every stored fingerprint uncomparable
- site/content/docs/migrate.mdx:310 — states the guard without these exceptions
## Required outcome
- A cut-off run doesn't leave the next `prod` unchecked. For example, store the fingerprint on each migration's row as it finishes, or carry the previous one forward.
- The docs say when `prod` can't compare, what it does then, and that `check` is the way to verify after a Reylo or server upgrade.
## Open questions for the human
- After an upgrade with migrations pending, should `prod` refuse and point to `check`/`sync-history`, rather than run unchecked? That costs friction on every minor upgrade, which is presumably why it goes on now.ℹ️ A migrations directory that doesn't exist makes dev rewind everything
At commands.rs:107, read_migrations returns no migrations for a directory that doesn't exist. dev then counts agree = 0 and rewinds every applied migration, dropping their tables and rows. reylo migrate dev --dir reylo-typo does this, and so does --dir --watch, where --dir takes --watch as its value. In a locked project the lock at the typo'd path is missing too, so dev_lock_problems returns early and nothing refuses. An empty directory can legitimately mean "no migrations on this branch". A directory that doesn't exist, on a database with applied history, is almost certainly a mistake, and dev (and reset) could refuse it.
dev undoes changes no migration made only where the pending migrations would run without them. prod stores the fingerprint on each migration's row as it finishes, and compares the server's major version alone. dev, reset and sync-history refuse a migrations directory that isn't there.
reset drops everything in the database and runs nothing; dev runs the migrations after it. pull writes each migration a database ran back into its file, as it ran. create says what it wrote, and the help lists the commands and the flags apart.
A terminal block holds a prompt, the command and what it printed, coloured as reylo colours it. A diff's lines carry a + or - of their own.
|
Replying to the review of 769a371, all fixed in 864f37e:
|
There was a problem hiding this comment.
Important
The new commits address all three concerns from the last review. One new gap: reylo migrate pull is now the fix every lock refusal names, but pull treats a file as matching under a looser comparison than the lock uses. For a migration that was only reformatted, pull reports success, changes nothing, and the refusal stays. This finding is in the body because GitHub can't anchor comments in engine/migrate/src/commands.rs.
Reviewed changes
This run reviewed the three commits since 769a371. It skipped what chore: merge main brought in.
- Kept what
syncmade when a draft is broken:devfirst replayskept + records + pendingon a clean shadow lane. If that fails, it refuses before it undoes anything. This resolves the earlier⚠️ . - Stored
prod's fingerprint per migration: each row gets the fingerprint as its migration finishes. A run that is cut off, or a later migration that leaves part of itself, is now still compared. The version in the fingerprint is the server's major version, and the docs now say when no comparison is possible. - Refused a migrations directory that doesn't exist:
devandsync-historyrefuse a directory that isn't there when the database has applied history. The CLI refuses a flag value that starts with--, so--dir --watchno longer works as a typo. - Made
resetonly drop:resetnow wipes the database, history included, and runs nothing. Every hint now saysreylo migrate resetfollowed byreylo migrate dev. - Added
reylo migrate pull: it writes each applied migration's text from the history table back into its file, and every Missing or Edited refusal now names it as the fix. - Docs: commands in
migrate.mdxare shown with their output in a new```terminalgrammar (site/lib/terminal.ts), with+/-gutters for diffs. Migration file names are no longer analyzed as statements. The skill is updated forresetandpull.
⚠️ pull doesn't fix the "changed after it was locked" refusal that names it
The lock calls a file Edited when lock::canonical of its text differs from what was locked. canonical drops comments and collapses runs of whitespace, but it keeps whether whitespace is there at all: (id int) and ( id int ) read as different. It also keeps text after #, /*! and --x as written. pull skips any file where same_statements holds, and that compares tokens with all whitespace and comments removed. So if a formatter reflows a locked migration, check and prod refuse it with "Take back the text a database ran: reylo migrate pull". Running pull then prints ✓ … holds every migration this database ran, as it ran it, writes nothing, and the next check refuses again.
Technical details
# pull's "already the same" test is looser than the lock's Edited test
## Affected sites
- engine/migrate/src/commands.rs:1564 — `pull` skips a file when `same_statements(&f.sql, &r.sql, d)`, a token comparison with every comment and all whitespace removed (`sql.rs` `Statement.tokens`)
- engine/migrate/src/commands.rs:586-607 — `links` marks `Link::Edited` when `lock::checksum(prev, name, normalized(&f.sql))` differs, i.e. over `lock::canonical`, which keeps single spaces between tokens and keeps text as written after `#`, `/*!`, `--x`, ambiguous block comments and escaped quotes (engine/migrate/src/lock.rs:85-195)
- engine/migrate/src/commands.rs:638 and :2586 — the `Link::Edited` refusals that now name `reylo migrate pull` as the fix
## Required outcome
- After `pull` from a database that ran a locked migration, that migration's lock link is `Ok` whenever the database ran the locked text. In particular a whitespace-only reflow (`create table a (id int);` → `create table a (\n id int\n);`) is written back.
- A scenario: lock a migration, reformat its file, see `check` refuse with `Edited`, run `pull`, and see `check` pass.
## Suggested approach (optional)
- In `pull`, leave a file as it is only when `lock::canonical(&normalized(&f.sql)) == lock::canonical(&normalized(&r.sql))`, the lock's own notion of "the same", rather than `same_statements`. A file that differs only in comments `canonical` drops is still left alone, as the doc comment says.claude-opus-5-5 ｜ 𝕏
init comes first, dev and prod share one heading, and so do lock and unlock. The help follows that order, and paints its headings, commands and flags, in a terminal and on the page.
There was a problem hiding this comment.
ℹ️ No new issues in this commit, just one minor docs suggestion inline. The
pulland lock-comparison gap from the last review is still open: this commit changes nothing inengine/migrate/src/commands.rsbeyond the visibility ofpaint.
Reviewed changes
This run reviewed the one commit since 95241c0.
- Reordered the help:
MIGRATE_USAGEnow lists the commands in the order they are used:initfirst,prodafterdev, thenresetandwipeaftercheck. The docs' copy matches, andtests/cli.test.tsnow looks for it in a```terminalblock. - Painted the help in a terminal:
painted_usage()makes headings bold, commands cyan, flags yellow and<…>values dim. It usescommands::paint, which is nowpuband has aboldstyle.main()decides whether to colour before dispatch, so piped output stays plain, and a unit test checks that the unpainted output equalsMIGRATE_USAGE. The regex covers every line of the help, including-h, --helpand--rename <old>=<new>. - Reordered the docs page: the sections in
migrate.mdxfollow the help's order.devandprodare now one section, and so arelockandunlock, which also holds the lock's YAML example. Each section now opens with a one-line summary. No link anywhere in the repository points to the anchors that went away. - Extended the
terminalgrammar:site/lib/terminal.tscolours a help block from itsUsage:line onward, and colours file trees' branches and their#notes.
claude-opus-5-5 ｜ 𝕏
| No migration was written. | ||
| ``` | ||
|
|
||
| ### `migrate` |
There was a problem hiding this comment.
Every other heading on this page is a command or a pair of commands (`lock` and `unlock`). But migrate isn't a subcommand: reylo migrate migrate doesn't exist, and a reader looking for dev or prod in the page's outline won't find either. The pair form you already use for the lock would keep both names visible.
| ### `migrate` | |
| ### `dev` and `prod` |
There was a problem hiding this comment.
Left as it is for now: this heading is deliberate, and what it should be depends on an open decision about whether the other commands stay under reylo migrate.
There was a problem hiding this comment.
ℹ️ No new issues. This commit fixes the gap between
pulland the lock that the review of 95241c0 raised. One earlier minor thread is still open: themigrateheading insite/content/docs/migrate.mdx:207, which this commit doesn't change.
Reviewed changes
This run reviewed the one commit since c60a61a.
- Made
pullcompare files the way the lock does:pullnow leaves a file as it is only whenlock::canonical(&normalized(…))of the file and of the history row agree.linkschecksums that same reading of the file, so any file the lock marksEditedthat the database ran is now written back. A reflow by a formatter gets written over, and a file that only gained a comment stays as it is. The doc comment says so, and the now-unuseddialect()call is gone. - Added
pull_takes_back_what_the_lock_refuses: in a strict project, the test reformats a locked migration. It checks thatcheckrefuses it and namesreylo migrate pull, thatpullwrites the original text back, and thatcheckthen passes. It also checks that a file whose only change is a leading comment is left as it is and still passes.
claude-opus-5-5 ｜ 𝕏
migrate.lock takes true or false, as reylo migrate lock and unlock set it, in place of migrate.history. The database is the type check's connection, so migrate.url is gone, and comments are always schema, so strict_comments is too. init adds the section the other commands need, and on an empty database writes only schema.sql. The page opens on what it does, then Basic usage.
There was a problem hiding this comment.
Important
The config cleanup holds together: Scope is gone everywhere it was used, old keys are now a parse error, and the docs and the skill match. One new path leaves a project stuck. If init fails after it has written its files, the migrate section is never added, so every command says to run init, and init refuses because schema.sql exists.
Reviewed changes
This run reviewed ce95354, the one commit since aa18bb6. It skipped what chore: merge main brought in.
- Made the lock a boolean:
migrate.lock: true | falsereplaceshistory: linear | loose.unlockwriteslock: falseandlockwriteslock: true.with_locknow only sets or inserts the line, so the "empty section becomesmigrate: {}" path is gone. - Dropped
migrate.urlandstrict_comments: the database is--url, else the type check'sdirect_url, elseDATABASE_URL. Comments are always compared.Scopeis removed from the catalog, the replay cache key and the fingerprint, which is nowcode_id:version:digest. Fingerprints stored in the old format can't be compared, so the firstprodafter the upgrade runs unchecked, as for any change tocatalog.rs. - Made
initstart a project:initruns without amigratesection, and on success the CLI appendsmigrate:\n lock: trueto.reylo.yaml, creating the file if needed. On a database with no schema (holds_a_schema, now shared withcheck) it writes only an emptyschema.sql, and the first migration iscreate's. Scenario and CLI tests cover both paths, on every dialect. - Docs:
migrate.mdxnow opens with a feature list and "Basic usage" starting frominit. The configuration sections and the skill list the new keys.
claude-opus-5-5 ｜ 𝕏
Dropping a table where foreign keys are on deletes its rows first, which fires the actions of the tables that reference it. A rebuild now rebuilds those tables too, at any depth, referencing the new one, so the file keeps every row on whatever runs it, with no pragma of its own.
There was a problem hiding this comment.
Important
Rebuilding the tables that reference a parent along with it is the right fix, and it keeps the file free of pragmas. But the tables are copied and dropped in breadth-first order, which is not a dependency order. A common schema shape (users, posts(user_id), comments(user_id, post_id)) still fails with FOREIGN KEY constraint failed when foreign keys are on, so the module doc's promise ("keeps every row on whatever runs it, foreign keys on or off") doesn't hold yet. Details inline.
Reviewed changes
This run reviewed a61e6ce, the one commit since 750c421.
- Rebuilt referencing tables with their parent: when SQLite rebuilds a table,
alter_tablenow also rebuilds every table that references it, at any depth (children_of). Each one is rebuilt from its current text, with itsREFERENCESpointing at the scratch tables. Rows are copied parents first, the old tables are dropped children first, and every table is then renamed back. So with foreign keys on, no drop fires a cascade or aset nullon rows that still matter. - Ordered what a rebuild takes down across tables: each
goesentry now carries its position in the schema. That lets the indexes, views and triggers of the whole family drop and come back in schema order, each once. - Gave the alterer both schemas: the new
Alterer::schemas(from, to)hook is called beforedecide. The SQLite alterer keeps both schemas so a table's statements can reach the tables that reference it. - Wrote a self-reference to the scratch table: a self-referencing table's rebuild now points at
new_<name>, which the rename turns back into its own name (sl26_self_reference_rebuild). - Tested it under foreign keys:
a_rebuild_keeps_every_row_where_foreign_keys_are_onruns a generated file on a connection with foreign keys on, inside a transaction. It covers cascade,set null, no action, a second level and a self-reference, and checks row counts,foreign_key_check, the AUTOINCREMENT counter and a view.
claude-opus-5-5 ｜ 𝕏
There was a problem hiding this comment.
ℹ️ No new issues. This commit fixes the
initdead end from the review of 750c421, and that thread is resolved. This review doesn't approve, because the earlier⚠️ thread onchildren_ofinengine/migrate/src/diff/sqlite.rsis still open. The SQLite family is still copied and dropped breadth-first, and this commit doesn't change that.
Reviewed changes
This run reviewed ffffc4f, the one commit since 083d158.
- Added the
migratesection wheneverinitwrote the files: the CLI now callsadd_migrate_sectionwheneveropts.schemaexists afterreylo_migrate::initreturns, whatever the result. Before, it only did so onOk(0). Ifsync_historyfails after the files are written, the project now has its section, soreylo migrate sync-historycan finish the job.add_migrate_sectiondoes nothing when a section is already there, so aninitrefused becauseschema.sqlexists doesn't add a secondmigrate:key. - Tested a stranded project:
tests/cli.test.tswrites an emptyreylo/schema.sqlwith no section and checks thatinitexits 1 with "schema.sql exists already". It also checks thatinitstill addsmigrate:\n lock: true\nand thatcheck --offlineno longer says there's no migrate section.
claude-opus-5-5 ｜ 𝕏
Each table is filled after, and dropped before, the tables it references, so a table that references the rebuilt one and a sibling keeps its rows with foreign keys on. Tables that reference each other have no such order, and defer their keys to the commit.
There was a problem hiding this comment.
✅ No new issues found. This commit fixes the ordering of the SQLite rebuild that the review of a61e6ce raised, and that thread is resolved. This review isn't an approval only because the earlier thread on the
### \migrate`heading insite/content/docs/migrate.mdx` is still open, and the author has said it waits on a decision.
Reviewed changes
This run reviewed 629ad11, the one commit since ffffc4f.
- Ordered the rebuilt family by its references:
children_ofstill finds the tables that reference the rebuilt one breadth-first. It then places each table only once every family table it references is placed, so rows are copied after what they reference and old tables are dropped before it. A table's reference to itself is never one of itsdeps, so it can't block placement. - Deferred the keys of tables that reference each other: when no order exists, either among the children or through the rebuilt table itself, the file now adds
pragma defer_foreign_keys = onwith a comment. The pragma lasts until the transaction commits. Reylo's own runner wraps each SQLite migration inbegin, and the replay cache's session key readsdefer_foreign_keysonly after the commit has reset it. - Tested both shapes with foreign keys on:
a_linksreferences the parent and alsokids, a sibling that sorts after it. That is the shape the earlier thread reproduced.a_rebuild_keeps_tables_that_reference_each_otherruns a two-table cycle withon delete cascade. It checksforeign_key_check, that both references survive, and that a laterdevand dry run come out clean.
claude-opus-5-5 ｜ 𝕏
Adds migrations to Reylo:
reylo migrate, in the native binary, for Postgres, MySQL and SQLite. The schema is declared inreylo/schema.sql;reylo migrate create <name>writes each change as a plain.sqlmigration, tried on a shadow database before it is written;reylo migrate devruns them in development, andreylo migrate prodin production. The docs page issite/content/docs/migrate.mdx(/migrate).Commands
Each is one operation, with no flag that makes it another.
dev,sync,resetandwiperefuse to run whereCIis set.How it holds
reylo/reylo-lock.yaml, lists the migrations in the order they run, each with a checksum chained to the line before, and the checksum ofschema.sqlfrom the last time the migrations reached it. Two branches that add migrations conflict on it; a wrong "accept both" breaks the chain, offline. A file the lock lacks is a draft, which onlydevruns.reylo migrate unlockturns it off (lock: false).prodbuilds no shadow database. It stores a fingerprint of the schema after each run, refuses a database that no longer has it, and namescheck, which replays the history in shadow databases and says how it differs. A deploy runscheck, thenprod.sync-history, which proves the schema first, anddev, where the database already has a migration's change.createrefuses a plan that drops a table or column, or changes a column's type so values are lost without failing, until--dropnames it.Also
reylo/directory (dirin.reylo.yaml); the type check's cache moves from.reylo/toreylo/.cache/, which keeps itself out of git..reylo.yamlkeys: the top-leveldir, and themigratesection (shadow_url,lock), whichreylo migrate initadds.Tests
engine/migrate/tests/scenarios.rs: 237 scenarios on Postgres 18, MySQL 8.4 and SQLite (branches, lock conflicts, drift, cut-off statements, renames, lossy types, partitions, watch).engine/migrate/tests/diff*.rs), every crate's tests, the oracle, and the loader's tests with none skipped.