8000 Support for partition expression in checksum command lines by aadant · Pull Request #1118 · Altinity/clickhouse-sink-connector · GitHub
[go: up one dir, main page]
Skip to content

Support for partition expression in checksum command lines - #1118

Merged
aadant merged 2 commits into
developfrom
add_partition_expression_template
Aug 1, 2025
Merged

Support for partition expression in checksum command lines#1118
aadant merged 2 commits into
developfrom
add_partition_expression_template

Conversation

@aadant
@aadant aadant commented Jul 31, 2025 •
edited
Loading
Copy link
Copy Markdown
Collaborator

This PR is to automate the checksum based on partition dates (without specifying the partitioning key)

python db_compare/mysql_table_checksum.py --min_date_value "1900-01-01" --mysql_host $MYSQL_HOST --mysql_database $DATABASE --tables_regex "$TABLE" --where "{partition_expression}=$DATE_FORMATTED" --max_datetime_value "2299-12-31 00:00:00"  --binary_encoding base64 | grep -i checksum | awk '{print $11" "$13" "$15}' >mysql.txt
python db_compare/clickhouse_table_checksum.py --clickhouse_host $CH_HOST --clickhouse_user dba_aadant  --clickhouse_database $DATABASE --tables_regex "$TABLE" --where "{partition_expression}=\\'$DATE\\'"  --max_datetime_value "2299-12-31 00:00:00" --exclude_columns _version,is_deleted --sign_column "" |grep -i checksum | awk '{print $11" "$13" "$15}' > ch.txt
sort mysql.txt > mysql.sorted.txt
sort ch.txt > ch.sorted.txt
diff mysql.sorted.txt ch.sorted.txt

@aadant
aadant requested review from minguyen9988 and subkanthi July 31, 2025 21:34
@aadant
aadant merged commit e0e28ac into develop Aug 1, 2025
@aadant
aadant deleted the add_partition_expression_template branch August 1, 2025 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@subkanthi subkanthi Awaiting requested review from subkanthi
@minguyen9988 minguyen9988 Awaiting requested review from minguyen9988

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@aadant
0