Repository navigation
fix(python): re-check a differing table before reporting a checksum difference (Spec 13.06 section 3.7.3) - #1547
Merged
Conversation
…ifference (Spec 13.06 section 3.7.3) On a table written during the run the sides read it at different moments, and the MySQL side reads its PK chunks over separate connections, so one pass can differ although no row diverges. Seen on a scheduled run: 15 rows closed and 15 successors inserted mid-scan gave equal counts (27,368,866) and different md5s; the next run matched and the 30 rows are identical on both sides. Both drivers now run each table through verify_table: a DIFFERENT pass is checksummed again --recheck_differences more times (default 1), --recheck_delay_seconds apart (default 60). Earlier passes log the mismatch at INFO; only the last pass logs the WARNING 'Checksum difference' that scheduled jobs fail on, so a difference that persists still fails the job and one the re-check no longer sees does not. A re-check that errors is an ERROR. --recheck_differences 0 restores the single pass, and a caller that builds its own Namespace without the new attributes keeps it. D-13.06-36 (catch-up not established) stays open; this mitigates its noise. Tests: new db_compare/tests/test_checksum_recheck.py (legacy driver end to end with the scheduled job's flags; verify_table in both copies). db_compare, db_dump and db_load: 649 passed, 4 skipped. Mutation (re-check removed): 7 of the new tests fail. Co-Authored-By: Claude Opus 5.5 (1M context) [noreply@anthropic.com]
minguyen9988
added a commit
that referenced
this pull request
Oct 2, 2026
#1547 was squash-merged into 2.11.0 with the same content as its commit on this branch (40162c9), and #1537-#1543 were merged with no content change, so the two conflicting files keep this branch's version: 2.11.0's version of them plus this PR's change. Co-Authored-By: Claude Opus 5.5 (1M context) [noreply@anthropic.com]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem: on a table that is written during a checksum run, the sides read it at different moments, and the MySQL side reads its PK chunks over separate connections, each with its own snapshot. One pass can therefore report a difference although no row diverges. Observed on a scheduled run on 2026-10-01: the ClickHouse side finished at 17:45:30; 15 rows were closed (db_to set) at 17:45:35.9 and their 15 successors inserted from 17:45:36.7 to 17:45:38.4, while the MySQL side ran from 17:45:28 to 17:46:04 in 16 PK chunks. The chunk holding the closed rows saw the update and the last chunk did not see the inserts, so both sides reported 27,368,866 rows with different md5s. The next run matched, and the 30 rows hold identical values on both sides.
Change (both drivers, legacy and packaged):
Spec: 13.06 gains section 3.7.3, two CLI rows, the outcome-table row, the DIFFERENT logging rule in 3.8 and the new tests in 5.1. D-13.06-36 (replica catch-up is never established) stays open and is marked mitigated, not fixed: a table written so continuously that every pass races a write can still be reported DIFFERENT. Fixing that needs a source binlog position read in the same snapshot and a wait for the connector offset to pass it.
Verification:
🤖 Generated with Claude Code