Repository navigation
fix(cdc): lifecycle -- unconvertible ROW records are terminal, offset FIFO survives restarts, terminal failures terminate, validated offset edits, binlog_row_image preflight - #1477
Merged
Conversation
… as a heartbeat
A Debezium record that produced no ClickHouseStruct was always remembered as
the batch's lastControlRecord and, once the pipeline was quiescent, its offset
was committed (commitControlRecordOffset). That is correct for a heartbeat,
a transaction marker or a tombstone, which carry no row by contract. It was
silent data loss for a ROW record (value with an `op` field) whose parse
returned null (missing before/after section, unknown op) or threw (JSON
payload, converter gap): processEveryChangeRecord logged a WARN "skipping"
(or the catch-all logged the exception) and returned null, the batch loop
treated that null like a heartbeat's, and the durable source position moved
past a row that never reached ClickHouse. A restart did not redeliver it.
Now:
- isControlRecord: null value (Debezium tombstone) is a control record;
a non-null non-Struct value is not.
- processEveryChangeRecord wraps the parser call; a throw, or a null result
for a non-control record, is raised as RecordReplicationException, a new
type re-thrown ahead of the catch-all exactly like DDLReplicationException,
so it leaves handleBatch and halts the engine with the offset behind it.
- handleChangeEventBatch is the second line: a null struct may become
lastControlRecord only if isControlRecord(sr); otherwise it throws.
- A value that is neither Struct nor null is rejected before the version
sequence is touched, with a message naming the class and topic instead of
the bare ClassCastException it produced before.
Deliberately INVERTED tests (they pinned the defect):
- NullParsedRecordSkipTest -> NullParsedRowRecordIsTerminalTest: it asserted
"skipped deliberately at WARN, no NPE" for a row record parsing to null.
The skip was the loss. It now asserts RecordReplicationException from
processEveryChangeRecord, still without an NPE and with no "skipping" line.
- ControlRecordLogLevelTest.unparseableRowRecordStillWarns ->
unparseableRowRecordIsTerminal: same inversion at the batch-handler seam.
- UnparseableRecordProgressIT: its "WARN row-record skip" leg is removed
(that outcome is now a defect); the records that run produces are control
records and the remaining assertions are unchanged. Not runnable here
(Testcontainers); compiled only.
Specs: 01.06 s3.1 rewritten (it codified the skip) with the classification
table and redelivery behaviour; 10.04 s3.3 now covers both terminal types;
01.03 dispatch loop updated; CONSTITUTION s5.1 and Lean README I9 rows.
Lean: Snapshot.lean gains Rec/dispatch with unparsed_row_halts,
unparsed_row_never_committed, parsed_row_is_handoff and the counterexample
old_rule_commits_unparsed_row (lake build clean, zero sorry).
Tests: UnparseableRowRecordIsTerminalTest (7) -- op='c' parser throws and
op='u' parse null propagate RecordReplicationException with
committer.processed empty and batchesFinished == 0; non-Struct value;
heartbeat preceding the bad row not committed; tombstone and heartbeat still
commit; classifier table. Failing before the fix ("nothing was thrown"),
green after; mutation check (both layers disabled) turns 5 tests red.
…offset FIFO DebeziumOffsetManagement's handoff FIFO is static, but the embedded engine is restarted INSIDE the process by REST /restart, by /start after /stop, and by the restart monitor -- each builds a new DebeziumChangeEventCapture on the same FIFO. The old stop() shut the worker pool down FIRST (cancelling the periodic workers, abandoning every queued batch), closed the engine LAST, and never touched the FIFO. A unit the old engine had handed off but no worker had written stayed the FIFO head for the life of the JVM: every unit of the new engine parked behind it, no offset was ever acknowledged again, hasUnwrittenBatches() stayed true (no control-record commit; every DDL drain timed out into a restart loop), rows kept being inserted while the durable offset froze, and the parked units' record lists leaked. Now: - stop() closes the engine first (no more handoffs), stops the event thread, drains through the still-running pool bounded by stopDrainTimeoutMs (60 s), shuts the pool down, and only then calls DebeziumOffsetManagement.reset(), which abandons whatever is still outstanding (WARN with the count; stop() returns it). Nothing abandoned was acknowledged, so the next engine redelivers it from the last committed offset: redelivery, never loss, never a rolled-back offset. - setup() refuses (IllegalStateException) to start a new engine while hasUnwrittenBatches(): a restart path that skipped stop() fails loudly instead of starting a connector that can never acknowledge an offset. - Restart monitor: effectiveLastRecordTimestamp(inMemory, stored) replaces `if (storedOffsetsInTable == -1) lastRecordTimestamp = storedOffsetsInTable`, which adopted the stored offset timestamp only when it was the sentinel; a valid stored value was never used, so the monitor restarted the engine on every tick until the first record arrived (each restart going through the poisoning stop()). Specs: 01.01 s3.2/s3.3 rewritten with the ordered stop sequence and why; 09.01 s2 state note, new s3.8 (abandon, never poison), verification; CONSTITUTION I8 text and s5.1 row; Lean README. Lean: OffsetFifo.lean gains the `abandoned` field and `restart` event; the invariant is extended (completed subset of outstanding, acked/outstanding disjoint from abandoned) and re-proved; new theorems restart_quiescent, acked_never_rolled_back, abandoned_not_acked, the counterexample old_restart_poisons_fifo and restart_unblocks_next_engine; acked_downward_closed is now stated modulo abandoned sequences. lake build clean, zero sorry. Tests: EngineRestartFifoResetTest (5) -- a handoff never written, stop(), a new instance's heartbeat commits and its first unit is acknowledged with completedUnits empty; stop() leaves nothing outstanding/unwritten/parked; engine closed before the pool is shut down; in-flight work drained before the pool stops; setup() refuses while a unit is outstanding. RestartMonitorTimestampTest (3). Before the fix 5 + 1 failed (the ghost stayed outstanding; stored timestamp not adopted); green after; mutation check (old order, no reset) turns 4 of 5 red.
…is not terminal The engine's CompletionCallback recreated a failed engine until numRetries passed MAX_RETRIES -- and then returned. The JVM stayed up with replication stopped, the REST API answering and the metrics port open: no process-level signal for a supervisor or a liveness probe. numRetries was never reset, so a connector that had recovered from ten transient failures over its lifetime died silently on the eleventh. And the DDL drain aborted after a fixed 60 s, so a worker retrying a transient TOO_MANY_PARTS for longer turned into DDLReplicationException -> engine restart -> the same drain -> ... -> the budget spent -> a permanent stop, for a condition that would have cleared. Now: - handleEngineCompletion (the callback body, extracted so it can be tested without an engine) retries exactly errors.max.retries times (the old `<=` allowed one more), then treats the failure as TERMINAL: replication marked not running, a FATAL line, and unless exit.on.terminal.failure=false the process exits through terminalFailureHook (System::exit in production) with TERMINAL_FAILURE_EXIT_CODE=3. With the exit disabled the process stays up as a visible liveness failure (/status Replica_Running=false). - markEngineStarted (connectorStarted) resets numRetries. - drainBeforeDDL waits while the workers are alive: failIfWorkerDied on every poll (a dead worker is the one backlog that can never drain, and it aborts at once with the cause), a WARN naming the backlog every ddlDrainWarnIntervalMs (60 s, formerly the timeout DDL_DRAIN_TIMEOUT_MS), and the in-flight wait (awaitQuiescent) loops the same way. Only a dead worker or an interrupt aborts the attempt. - New property exit.on.terminal.failure (SinkConnectorLightWeightConfig), documented in doc/configuration.md. Deliberately INVERTED tests (they pinned the timeout abort): - DdlDrainDeadlockTest.testGenuinelyStuckQueueStillAborts -> testStuckQueueWithDeadWorkerAborts: an undrainable queue aborts promptly and only because its worker is dead, carrying the cause. - DdlDrainDeadlockTest.testUndrainableRoutedQueueAborts -> testUndrainableRoutedQueueWithDeadWorkerAborts: same for a routed backlog, message still names the routed backlog. - New testUndrainableQueueWithLiveWorkersKeepsWaiting: with a live worker the drain keeps waiting past several warn intervals, logs the WARN, and ends only on interrupt. Specs: 10.04 s3.4 note and new s3.5 (retry budget, then exit); 06.01 s2, s3.2 (liveness-bounded wait), s4, s5; doc/configuration.md. Tests: TerminalFailureExitTest (4) -- the exit hook fires exactly once after MAX_RETRIES restarts and not before; a successful start resets the budget; exit disabled logs FATAL and reports Replica_Running=false; success is a no-op. Before the fix 3 of 4 failed (one extra restart, no exit, no reset) and the drain tests aborted on the timeout; green after; mutation check (no reset, no exit, timeout abort restored) turns them red.
…ffset as the newest row The REST position edits overlaid whatever they were given onto the stored offset JSON: the binlog position was stored as a string (Debezium fails at the next start on anything non-numeric), a new file/position left the stored gtids in place (Debezium resumes from a GTID set when one is present, so the edit was silently ignored) and left the stored row/event skip counters in place (they belong to the OLD position; kept, Debezium skips that many rows of the first event at the NEW position -- silent loss), and an empty edit was accepted. A PostgreSQL LSN in X/Y form was parsed as Y alone. The base row was read with a plain `select offset_val ... where offset_key=...`, which on a ReplacingMergeTree with unmerged parts returns an arbitrary one of several checkpoints, so the edit could be applied on top of a stale base. Now (DebeziumOffsetStorage): - updateBinLogInformation: file and position must come together; the position must be a non-negative integer and is stored as a number; a file/position edit resets row/event to 0 and, unless a gtid is given in the same request, removes gtids; a GTID-only edit keeps file/position; an empty edit is refused (IllegalArgumentException). - updateLsnInformation: lsn required; X/Y = (X << 32) | Y; malformed refused. - offsetValueQuery: `FINAL ... order by record_insert_ts desc, record_insert_seq desc limit 1`; no FINAL when the configured offset DDL declares KeeperMap (one row per key, FINAL rejected). - DebeziumEmbeddedRestApi: /binlog and /lsn answer 400 with the reason on a malformed edit and while replication is running; connections closed via try-with-resources; numeric JSON positions accepted. Deliberately INVERTED test: DebeziumChangeEventCaptureTest .testUpdateBingLogInformation pinned `"pos":"1222"` (string) and the stale `"row":1,"event":2`; it now expects a numeric position and both counters reset, comparing parsed fields instead of a whole-string match. Spec 09.03: s2 mapping, s3.2 read (newest row, KeeperMap note), new s3.4 validation table and CLI note, s5 verification. Tests: OffsetEditValidationTest (11). Before the fix 10 of 11 failed (string position, gtids kept, counters kept, nothing refused, high LSN word dropped, plain select); green after; mutation check (validation and FINAL/ordering removed) turns them red.
…ow_image is not FULL
Nothing checked binlog_row_image. The connector turns every row event into
a full-row insert whose newest version replaces the previous one column for
column, which is only correct when every row event carries every column.
With MINIMAL an UPDATE's after-image omits the columns the statement did
not change, so every update of every table was replicated with those
columns as NULL (or a ClickHouse default) -- a value-level divergence on
every update with row counts intact; with NOBLOB the same for every
BLOB/TEXT column. Nothing downstream can recover a value the source never
logged, so unlike the keyless-table report (which names specific tables an
operator may knowingly accept, and whose class contract is "never blocks
startup", structurally tested) this is all-or-nothing.
New BinlogRowImagePreflight, wired in DebeziumChangeEventCapture.setup()
right after KeylessTablePreflight.check:
- reads SELECT @@GLOBAL.binlog_row_image (the SHOW GLOBAL VARIABLES LIKE
'binlog_row_image' value, issued as a SELECT so it passes the existing
SELECT-only read-only allowlist) on a read-only connection built with the
keyless check's TLS-aware JDBC URL;
- refuses to start (IllegalStateException, ERROR banner naming the value
and the fix `SET GLOBAL binlog_row_image = FULL`) when the value is
readable and not FULL;
- an unreadable value (unreachable, permission denied) is a WARN and the
start proceeds, as for the keyless check; non-MySQL connectors pass;
- binlog.row.image.check.skip=true lets a non-FULL source through with a
WARN banner on every start.
Specs: 01.01 s2 mapping, s3.2 step 3 (row-image preflight paragraph), s5;
10.04 new s3.6; doc/configuration.md.
Tests: BinlogRowImagePreflightTest (7) -- a dynamic-proxy JDBC Connection
stub answering MINIMAL / NOBLOB is refused with the variable, value and fix
in the message; FULL (any case) passes; an unreadable source warns and
passes; the skip property is loud; non-MySQL and unreachable sources pass
through; the query passes the read-only allowlist. Against a log-only
scaffold 4 of 7 failed ("nothing was thrown"); green after; mutation check
(refusal replaced by a log line) turns them red.
Debezium's enable.time.adjuster defaults to true: a two-digit year -- and, on the MySQL connector, any year below 100 -- is remapped into 1970-2069, so a source DATE/DATETIME of 0001-01-01 arrived as 2001-01-01. That is a value-level divergence with row counts intact, decided by the connector on the source's behalf. Only the Ansible deployment template disabled it; the JAR's bundled defaults, the Docker configurations and every hand-written configuration ran with the adjuster on. DebeziumChangeEventCapture.setupDebeziumEventCapture now calls ensureTimeAdjusterDisabled(props) next to the column.propagate.source.type default: absent or blank -> "false" (INFO); an explicit value is left alone, an explicit true is logged at WARN. Spec 07.03: s2 mapping, new s3.3, s5; doc/configuration.md row. Tests: TimeAdjusterDefaultTest (4) -- absent and blank are forced to false, an explicit true wins, null tolerated. Against a no-op seam 2 of 4 failed; green after; mutation check (setProperty removed) turns them red.
… live and scoped to table/database drops Two defects on the sink DDL path, both in checkIfDDLNeedsToBeIgnored / performDDLOperation: 1. No table filter existed. Debezium emits a schema-change event for every table of the captured databases unless schema.history.internal.store.only.captured.tables.ddl=true, and the sink applied every one of them: a CREATE TABLE outside table.include.list created a spurious ClickHouse table, and an ALTER on such a table failed on the missing target (Code: 60) and, being terminal, halted the whole pipeline for a table nobody asked to replicate. New DdlCaptureFilter applies database/table include/exclude lists with Debezium's semantics (comma-separated regexes, full case-insensitive match, include list wins) to the record's SOURCE database (key databaseName/db, else source.db) and the tables in tableChanges / source.table / both sides of a rename. A multi-table DDL is kept when any table is captured; a table-less DDL is judged by the database lists; an unknown database is never filtered; an uncompilable exclude pattern is ignored (never drop a DDL by guessing). 2. disable.drop.truncate was dead: it was tested BEFORE parseSql computed the flag it reads, so the flag was always false; and the flag was raised by ANY DROP token, so had it worked it would have swallowed ALTER TABLE ... DROP COLUMN and DROP INDEX too. It is now evaluated AFTER the parse, on the statement kind: the MySQL listener sets it in enterDropTable / enterTruncateTable / enterDropDatabase, and both parser services' token helpers test the first significant tokens (DROP TABLE|TEMPORARY| DATABASE|SCHEMA, TRUNCATE). Skipping is logged at WARN and documented as a deliberate, operator-chosen divergence (ClickHouse keeps what the source removed). Spec 06.08: s2 mapping, new s3.3 (which DDL is ignored, rules 1-5), s4, s5; doc/configuration.md rows for disable.drop.truncate and store.only.captured.tables.ddl. Tests: DdlCaptureFilterTest (8), DdlIgnoreRulesTest (6). Before the fix 9 failed and 1 errored (everything captured; DROP COLUMN flagged; DROP TABLE with the property set went to execution); green after; mutation check (filter returns true, flag from any DROP token, post-parse check removed) turns them red. Destructive-Op-Check: sites=21; result=pass
…tch halts; ignore_delete announced as loss by design Configuration hygiene, all in the direction of 'a setting that looks set must be read, and a lossy setting must never be a default': - deploy/ansible-systemd defaulted schema.history.internal.skip.unparseable.ddl to true for every connector. With it Debezium silently discards DDL it cannot parse, so later rows of that table are decoded against a stale schema. Debezium's default is false; the role now defaults to false and documents enabling it as a deliberate per-connector override (defaults, template comment, README example). - The template emitted keys the connector never reads: clickhouse.table.engine, clickhouse.table.sign.column, clickhouse.table.version.column (removed), ignore.delete (the key is ignore_delete; renamed) and the deprecated clickhouse.server.database (removed; the database name is per source database / override map). It also emitted Debezium's max.queue.size where the comment described the sink's queue; it now emits sink.connector.max.queue.size (overridable via sink_connector.max_queue_size). - DbWriter's constructor and autoCreateTable caught Exception around ClickHouseAutoCreateTable, which raises ColumnTypeOverrideMismatchException precisely so the connector halts; both now re-throw it ahead of the generic catch, so a writer is never built against a type the operator declared wrong. - ignore_delete=true is loss by design (rows removed at the source stay visible forever). It is still honoured, but the config constructor logs one WARN per JVM naming the divergence, and doc/configuration.md says so. Specs: 10.04 new s3.7 (loss-by-design options) and s3.8 (writer halts on the mismatch); 08.05 new s3.3. Tests: IgnoreDeleteWarningTest (2) pins the predicate behind the WARN; red when the predicate is mutated to false. The DbWriter change has no unit test: building a DbWriter needs a live ClickHouse (DbWriterTest is Testcontainers-based). The template was rendered (minimal and optional-keys inventories, StrictUndefined) before and after; both parse as YAML with no duplicate keys and the diff is exactly the changes above. Jinja-Render-Check: rendered=1; formats=yaml; result=pass
…s; fix disable.drop.truncate swallowing snapshot DDL Resolves the four conflicts of merging the current 2.11.0 head into the branch, all kept-both: DbWriter.java both init catch clauses (ColumnTypeOverrideMismatchException rethrown first, then IllegalStateException from #1476 logged and rethrown); Lean README lists OffsetFifo + DdlTranslation + BatchOrder; spec 10.04 unions both verification lists; spec 07.03 keeps #1476's out-of-range 3.3 and renumbers this branch's enable.time.adjuster section to 3.4, with its code cross-references (DebeziumChangeEventCapture, TimeAdjusterDefaultTest) updated to 3.4. CI fix (java-tests-lightweight, DateTimeWithUserProvidedTimeZoneSchemaOnlyIT): this branch (794f1f2) made disable.drop.truncate live for the first time. The IT sets disable.drop.truncate=true with enable.snapshot.ddl=true, so the snapshot-phase schema-bootstrap DDL (a table drop-then-recreate) was suppressed, leaving the pre-created target table (Minimum_Value DateTime, min 1970-01-01) in place instead of the DateTime64 the snapshot recreate would build (min 1900-01-01 = DATETIME_MIN); the assertion at line 152 saw 1970-01-01 and failed. Fix: disable.drop.truncate suppresses STREAMING drops only; snapshot-phase DDL (isSnapshotDDL(sr)) is exempt, because a snapshot recreate is schema initialisation, not a source-initiated data drop during replication. Suppressing it would freeze a stale target schema and make ClickHouse diverge from MySQL, the opposite of the option's purpose. Spec 06.08 section 3.5 documents the snapshot exemption; the verification reference is corrected to 3.5. New unit test DdlIgnoreRulesTest.disableDropTruncateExemptsSnapshotDdl: a snapshot drop reaches execution despite disable.drop.truncate=true, while a streaming drop is still frozen. Mutation: dropping the isSnapshotDDL guard makes the snapshot assertion red. Spec validator PASSED (59 specs, 463 citations). DdlIgnoreRulesTest 7/7. Full sink-connector unit suite 541 run, 0 failures, 20 errors all Docker-environment (Testcontainers, no daemon here; identical to base). All 21 destructive sites are statement text parsed/classified/logged or test-code literals (no execution against any live database); the code sites carry #1476's DESTRUCTIVE: markers, unchanged by this merge. Destructive-Op-Check: sites=21; result=pass Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
Author
|
Pushed 7643932: merged the current 2.11.0 head (40752a4, PR 1476) into the branch to resolve the conflict, and fixed the red java-tests-lightweight job. Conflict resolution (all kept-both): DbWriter's two init catch clauses, the Lean README module list, spec 10.04's verification list, and spec 07.03 (the out-of-range section from 1476 kept, this branch's enable.time.adjuster section renumbered to 3.4 with its code cross-references updated). CI root cause: this branch made disable.drop.truncate live for the first time, and DateTimeWithUserProvidedTimeZoneSchemaOnlyIT sets disable.drop.truncate=true together with enable.snapshot.ddl=true. The snapshot-phase schema bootstrap (a table drop-then-recreate Debezium emits for each captured table) was being suppressed, so the pre-created target table (Minimum_Value as DateTime, min 1970-01-01) survived instead of being replaced by the DateTime64 the snapshot recreate builds (min 1900-01-01 = DATETIME_MIN); the assertion at line 152 saw 1970-01-01 and failed. Fix: disable.drop.truncate now suppresses STREAMING drops only; snapshot-phase DDL (isSnapshotDDL) is exempt, because a snapshot recreate is schema initialisation, not a source-initiated data drop, and suppressing it would freeze a stale schema and diverge from MySQL. Spec 06.08 section 3.5 updated; new unit test DdlIgnoreRulesTest.disableDropTruncateExemptsSnapshotDdl (mutation-checked); DdlIgnoreRulesTest 7/7; spec validator PASSED; full sink-connector unit suite 541 run, 0 failures, 20 Docker-environment errors identical to base. |
minguyen9988
added a commit
that referenced
this pull request
Sep 22, 2026
…-commit-order Resolves eight conflicts of merging the current 2.11.0 head (now containing #1475/#1476/#1477) into the version-floor branch. Additive/kept-both in the Lean and spec files: both BatchOrder (#1476) and VersionFloor (this branch) listed in lakefile.lean, formal_specs/lean/README.md, CONSTITUTION.md, and spec 11.03 (numbered proof list renumbered so BatchOrder is 13 and VersionFloor 14-16); spec 04.04 keeps #1476's collapsing/grouping bullets and this branch's composite-PK bullet; spec 01.06 keeps #1477's terminal classifier plus this branch's heartbeat-sequence-state test. The one semantic conflict is the version-assignment hot loop of DebeziumChangeEventCapture. Both behaviours are kept: #1478's VersionAssignment machinery (nextVersionAssignment + coverAssignedVersion, control records excluded from the sequence, chStruct versioned from assignment.sequenceNumber/effectiveTs) AND #1477's loud-failure path (rejectUnrepresentableValue before the sequence; parse() wrapped so a parser throw becomes RecordReplicationException; a null-parsed ROW record is terminal, not a WARN skip; the RecordReplicationException/DDLReplicationException rethrows ahead of the catch-all). NullParsedRowRecordIsTerminalTest keeps its terminal assertion, adapted to the VersionAssignment reflective signature. Verification: lightweight compile clean; NullParsedRowRecordIsTerminalTest + ControlRecordLogLevelTest + DebeziumChangeEventCaptureTest + UnparseableRowRecordIsTerminalTest 38/38; spec validator PASSED (59 specs, 519 citations); lake build completed successfully (BatchOrder + VersionFloor both built, 0 sorry); full sink-connector unit suite 547 run, 0 failures, 20 errors all Docker-environment (identical to base). config.yml.j2 rendered (minimal + full variants) and YAML-linted with duplicate-key detection: no unrendered markers, no duplicate keys. All 42 destructive sites in this merge are #1475/#1476/#1477 code already carrying DESTRUCTIVE: markers or test-code literals (statement text parsed/classified/logged, no execution against any live database), unchanged by this merge. Jinja-Render-Check: rendered=2; formats=yaml; result=pass Destructive-Op-Check: sites=42; result=pass Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
minguyen9988
added a commit
that referenced
this pull request
Sep 22, 2026
…ypes; fix inherited DdlIgnoreRulesTest signature Resolves nine conflicts of m 8000 erging the current 2.11.0 head (now containing #1475/#1476/#1477/#1478) into the ddl-translation branch. MySqlDDLParserListenerImpl.java: both sides added independent fields (this branch's PK-change/ddlEventTimestamp fields and #1477's dropOrTruncateStatement field + isDropOrTruncateStatement method) - kept both. Lean lakefile/README, CONSTITUTION and spec 11.03 list all of BatchOrder (#1476), VersionFloor (#1478) and this branch's CreateTable. spec 07.01 keeps this branch's DDL-path unsigned rule (§3.2) and #1478's record-path untyped-Int64 report (renumbered §3.3). spec 07.02 keeps this branch's IEEE section plus #1475's value-comparison caveat. spec 06.05 keeps both verification sets including #1478's RecordSchemaVsDdlTypeAgreementTest bullet. spec 07.06 (spatial) is a genuine two-path merge: the record-schema path keeps POINT/POLYGON native (#1475), the DDL path types every spatial column String/WKB-hex (this branch, new §3.4); they agree on every non-polygon type and the RecordSchemaVsDdlTypeAgreementTest does not cover spatial, so the code coexists. CI fix: 2.11.0 itself is internally inconsistent - #1478 changed processEveryChangeRecord's last parameter to VersionAssignment but DdlIgnoreRulesTest (added by #1477) still reflected long.class, so disableDropTruncateIsScopedAndLive and disableDropTruncateExemptsSnapshotDdl threw NoSuchMethodException. This is the java-tests-lightweight failure the branch inherits through the merge. Fixed the reflective lookup and call to use VersionAssignment (mirroring NullParsedRowRecordIsTerminalTest). The same one-line fix should be back-ported to 2.11.0 directly. Verification: spec validator PASSED (59 specs, 534 citations); lightweight compiles; DdlIgnoreRulesTest + MySqlDDLParserListenerImplTest + DataTypeConverterTest + CreateTableNoKeySortKeyTest + CreateTableUniqueKeySortKeyTest + NullParsedRowRecordIsTerminalTest 175/175 (1 Docker skip); full lightweight unit suite 556 run, 0 failures, 62 errors all Testcontainers/Docker (no daemon here). config.yml.j2 rendered (minimal + full) and YAML-linted, no duplicate keys. All destructive sites in this merge are #1475/#1476/#1477/#1478 code with DESTRUCTIVE: markers or test-code literals, unchanged by this merge. Jinja-Render-Check: rendered=2; formats=yaml; result=pass Destructive-Op-Check: sites=42; result=pass Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Base: 2.11.0 at 5780b62 (after 1470-1474). Part of the replication-correctness audit; spec-first, failing-then-passing test and mutation check per finding; Lean Snapshot and OffsetFifo models extended.
Verification: lightweight unit suite green; lake build clean, 0 sorry; spec validator passes. Docker-backed ITs rely on CI.
Merge note: see fix-writer-value-fidelity for the suggested order of the six audit PRs; forward-merge 2.11.0 after each merge to clear the shared registration files.