8000 Remove the previous coordinator from JDBC_PING2 on view change by ahus1 · Pull Request #55248 · keycloak/keycloak · GitHub
[go: up one dir, main page]

Skip to content

Remove the previous coordinator from JDBC_PING2 on view change - #55248

Draft
ahus1 wants to merge 1 commit into
keycloak:mainfrom
ahus1:is-55236-jdbc-ping-remove-left-coordinator
Draft

ahus1 wants to merge 1 commit into
keycloak:mainfrom
ahus1:is-55236-jdbc-ping-remove-left-coordinator

Conversation

@ahus1
@ahus1 ahus1 commented Nov 16, 2026
Copy link
Copy Markdown
Member

Closes #55236

Deleting the row of the previous coordinator lets the surviving node win the split-brain decision right away, instead of waiting up to 60 seconds for the row to become stale.

Decisions and assumptions

  • Only the previous coordinator's row is removed, and only by the new coordinator, and only if it is no longer in the new view. Rows of other leaving members are not coord = true and can't win the split-brain decision.
  • If the removed member is still alive, for example in a real network partition, it writes its row again. The usual tie-break applies then: the older node, with the lower address, wins.
  • This shifts which side errs: previously a crashed coordinator caused a 40 to 70 second outage with no ready node. Now a real partition has a short window of a few seconds, up to one health-check interval, where both sides may look healthy. Split-brain detection is polling-based anyway, and crashes are more common than partitions.

Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Deleting a live former coordinator’s row can temporarily select the smaller partition as the winner, requiring prompt republication and a live-partition regression test rather than reliance on periodic refresh.

1 open issue
What changed in this PR

This PR addresses #55236 by removing a departed coordinator’s JDBC_PING2 row, allowing surviving nodes to recover readiness without waiting for that row to become stale.

Changes:

  • Deletes the previous coordinator’s row when the local node is coordinator and the previous coordinator is absent from the new view.
  • Adds a regression test verifying removal while preserving an unrelated coordinator’s row.
  • Adds a separate design draft for consolidating database-backed session handling.
File Description
model/​infinispan/​src/​test/​java/​org/​keycloak/​jgroups/​protocol/​JdbcPing2Test.java Tests departed-coordinator cleanup and preservation of unrelated rows.
model/​infinispan/​src/​main/​java/​org/​keycloak/​jgroups/​protocol/​KEYCLOAK_JDBC_PING2.java Removes the departed coordinator’s database row during view changes.
model/​infinispan/​session-storage-refactoring.md Documents a separate session-storage proposal and its transaction, isolation, and validation requirements.

Review effort: Balanced

Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The proposal relies on unavailable reference documents and incorrectly treats embedded-session binary-storage isolation as an existing guarantee.

2 open issues
1 resolved since last review

Review effort: Balanced

Closes keycloak#55236

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
@ahus1
ahus1 force-pushed the is-55236-jdbc-ping-remove-left-coordinator branch from 5b3b57a to f6e31fa Compare November 16, 2026 20:13
@ahus1
ahus1 requested a balanced review from Copilot November 16, 2026 20:14
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the departed-coordinator database-row cleanup change.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.model.infinispan.EmbeddedInfinispanSplitBrainTest#testRevokedTokensReloadedOnMergeEvent

Keycloak CI - Store Model Tests

java.lang.AssertionError: Execution didn't complete: java.lang.RuntimeException: org.awaitility.core.ConditionTimeoutException: Condition with alias 'Persisted revocation was not restored after the view merge' didn't complete within 30 seconds because condition with Lambda expression in org.keycloak.testsuite.model.infinispan.EmbeddedInfinispanSplitBrainTest was not fulfilled.
	at org.junit.Assert.fail(Assert.java:89)
	at org.keycloak.testsuite.model.KeycloakModelTest.inIndependentFactories(KeycloakModelTest.java:448)
	at org.keycloak.testsuite.model.infinispan.EmbeddedInfinispanSplitBrainTest.testRevokedTokensReloadedOnMergeEvent(EmbeddedInfinispanSplitBrainTest.java:125)

Report flaky test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Two-node cluster: killing the JDBC_PING2 coordinator makes the survivor report split-brain and stay unready for ~40–70 s

2 participants

0