8000 test: fix flaky Keeper reconnect check in test_010064 by Slach · Pull Request #2143 · Altinity/clickhouse-operator · GitHub
[go: up one dir, main page]

Skip to content

test: fix flaky Keeper reconnect check in test_010064 - #2143

Open
Slach wants to merge 1 commit into
0.28.0from
fix/test-010064-keeper-reconnect-check
Open

Slach wants to merge 1 commit into
0.28.0from
fix/test-010064-keeper-reconnect-check

Conversation

@Slach
@Slach Slach commented Nov 6, 2026 •
8000
Copy link
Copy Markdown
Collaborator

test_010064 fails at CHI does not reconnect to Keeper even though nothing reconnected, e.g.

In both runs the operator log shows no reconnect: the CHK reconcile had no work, the CHI reconcile was skipped ("resolved zookeeper endpoints unchanged"), and the Keeper pod was not restarted.

system.zookeeper_connection.connected_time is not stored. ClickHouse computes it at query time as time(nullptr) - zookeeper->getSessionUptime() (StorageSystemZooKeeperConnection.cpp). Both values are whole seconds, so it jitters by ±1s within the same session.

The check now compares client_id (the Keeper session id) instead. It changes when ClickHouse opens a new session and stays the same otherwise. The rescale step further down only checks that ClickHouse is still connected, so it is unchanged.

Checked locally on orbstack (--native, operator :dev built from this commit, GitSHA a3344abde): test_010064 passed twice, 94/94 and 93/93 steps ok. On 26.3.33.10001.altinitystable, the two replicas reported client_id 1 and 2.

…est_010064

connected_time in system.zookeeper_connection is computed at query time as
now() - session uptime, both in whole seconds, so it jitters by 1s within the
same session and made the "CHI does not reconnect to Keeper" check flaky.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Slach
Slach requested a review from alex-zaitsev November 6, 2026 09:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0