8000 fix(inkless:switch): keep switched partitions eligible for ISR shrink by jeqo · Pull Request #756 · aiven/inkless · GitHub
[go: up one dir, main page]

Skip to content

fix(inkless:switch): keep switched partitions eligible for ISR shrink - #756

Closed
jeqo wants to merge 3 commits into
mainfrom
jeqo/shrink-switched-isr
Closed

jeqo wants to merge 3 commits into
mainfrom
jeqo/shrink-switched-isr

Conversation

@jeqo
@jeqo jeqo commented Aug 18, 2026 •
Copy link
Copy Markdown
Contributor

maybeShrinkIsr skipped every diskless topic, which includes a partition switched from classic. Those keep records below classicToDisklessStartOffset in the replicas' local logs, so a follower that stalls before reaching the seal is genuinely behind. Nothing removed it from ISR either: the controller shrinks only on fencing, and the seal is taken at the leader's LEO, so a lagging follower does not hold the switch back. It stayed in ISR without the prefix and was electable.

Switched partitions are now shrunk. Three things the shrink alone gets wrong:

  • Lag is judged against the seal, not the leader's LEO. Past a committed seal the local log grows only by consolidation, which every replica appends from object storage through DisklessLeaderEndPoint -- a local fetch that never reaches the leader. The leader's recorded state for a healthy consolidating follower therefore stops at the seal while its own LEO keeps advancing, so comparing the two would shrink the entire ISR out, with nothing to re-expand it (expansion also waits for a follower fetch). Partition takes the ceiling as an optional argument, defaulted off, so classic partitions are unaffected.
  • A catch-up fetch is scheduled when the leader moves. A new leader resets an in-ISR follower's recorded state to UNKNOWN, and a follower at the seal starts no fetcher, so the lag grace period would expire and the follower would be shrunk and re-admitted on every leader change.
  • A consolidating follower is diverted to the classic fetcher for one round on a leader change. Consolidation reads from object storage, so a new leader never sees a fetch from that replica and keeps the UNKNOWN state, which the shrink then acts on: the ISR collapses to the leader alone on every rolling restart, with nothing to re-expand it. One classic fetch records the position; the fetcher self-evicts at the seal and hands the partition back to consolidation. The diverted partition also leaves the consolidation fetcher manager, so two fetchers never append to the same log.

Only partitions with no committed seal stay exempt: born-diskless, or a switch aborted through AlterDisklessSwitch. Neither replicates a classic prefix from this leader.

Commits

  • fix(inkless:switch): keep switched partitions eligible for ISR shrink -- the shrink predicate, the seal-capped lag comparison, and the leader-change catch-up fetch, with tests.
  • test(inkless:switch): assert the catch-up fetch target on leader change -- test-only: the leader-change regression test picked a broker that ClusterImageTest.IMAGE1 does not register, so the leader endpoint never resolved and the assertion held on an empty fetcher map. It now moves the leader between two registered brokers and asserts the exact InitialFetchState.
  • fix(inkless:switch): re-establish leader fetch evidence for consolidating followers -- the classic-fetcher diversion on a leader change, and the ceiling applied to both sides of the lag comparison.

See the commit messages for the reasoning behind each.

Operator impact

A switched partition whose follower never replicated the classic prefix now shows up as under-replicated and is logged by the usual Shrinking ISR from ... to ... line, instead of silently staying in ISR. No new config or metric.

Test plan

Unit tests in ReplicaManagerInklessTest cover: a switched partition shrinking a follower that never reached the seal while a born-diskless partition is left alone; a consolidating switched partition keeping a follower recorded at the seal in ISR while the leader's log end offset advances past it; and the catch-up fetch scheduled on a leader change, asserted against the expected InitialFetchState. Two further tests cover the consolidating leader change: the follower is scheduled on the classic fetcher rather than handed to the reconciler, and a follower recorded beyond 8000 the seal stays in ISR. Each fails without its production change. The upstream PartitionTest suite passes unchanged, which covers the classic path through the new default argument.

Relation to #757

ISR correctness for a switched partition has two legs, and these PRs are one each. #757 closes admission: changePartitionReassignment applied the target replica set in one step for any diskless topic, so a replica added by a reassignment entered ISR holding none of the classic prefix. This PR closes recovery: a replica that ended up in ISR without the prefix -- however it got there -- was never lag-shrunk out. They are independent (either can merge first) and complementary: with only #757, a replica that stalls mid-switch stays in ISR forever; with only this PR, a wrongly admitted replica is electable until the lag timeout expires.

Both, together with the unfence guard from #754, key on classicToDisklessStartOffset == -1 to mean "born-diskless". That reading is imprecise for a partition whose switch was aborted via AlterDisklessSwitch(-1) while diskless.enable is still true -- it keeps a local classic prefix. Pre-existing (the predicate this replaces, !isDisklessTopic, skipped that partition too) and not fixable at any single one of these sites, since LogConfig.validateDisklessTransition forbids disabling diskless afterwards, which makes that state terminal. Tracked as a follow-up.

@jeqo
jeqo requested a balanced review from Copilot August 18, 2026 09:58
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables ISR shrinking for switched diskless partitions while preserving the exemption for born-diskless partitions.

Changes:

  • Refines diskless ISR shrink eligibility.
  • Starts a catch-up fetch after leader changes.
  • Adds regression tests for both behaviors.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
core/src/main/scala/kafka/server/ReplicaManager.scala Updates ISR shrinking and follower fetch scheduling.
core/src/test/scala/unit/kafka/server/ReplicaManagerInklessTest.scala Adds ISR and leader-change regression tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jeqo
jeqo requested a balanced review from Copilot August 18, 2026 10:35
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

core/src/main/scala/kafka/server/ReplicaManager.scala:3254

  • Switched consolidating partitions also satisfy !isBornDiskless, but their leader LEO advances beyond the seal while followers hand off at the seal to ConsolidationFetcher. That fetch path uses FUTURE_LOCAL_REPLICA_ID (DisklessLeaderEndPoint.scala:92-101), so it never refreshes the leader's follower ReplicaState (Partition.scala:1444-1460). Once the lag timeout expires, Partition.maybeShrinkIsr compares the stale seal offset with the leader's advancing LEO and removes healthy consolidating followers from ISR. Please preserve the liveness-based ISR behavior for the consolidated suffix, or evaluate lag only against the committed seal, and cover this mode with a regression test.
      val isBornDiskless = _inklessMetadataView.isDisklessTopic(topicPartition.topic()) &&
        _inklessMetadataView.getClassicToDisklessStartOffset(topicPartition) ==
          PartitionRegistration.NO_CLASSIC_TO_DISKLESS_START_OFFSET
      if (!isBornDiskless)

core/src/test/scala/unit/kafka/server/ReplicaManagerInklessTest.scala:7847

  • This verifies only that the switched partition was submitted; Mockito still permits an additional submission for bornDiskless, so a regression that shrinks both partitions would pass. Add an explicit negative verification for the born-diskless topic.
      verify(alterPartitionManager, times(1)).submit(
        ArgumentMatchers.eq(new org.apache.kafka.server.common.TopicIdPartition(
          switched.topicId(), switched.partition())),
        argThat[LeaderAndIsr](!_.isr.contains(followerId)))

@jeqo
jeqo force-pushed the jeqo/shrink-switched-isr branch 2 times, most recently from e4af171 to 8f5cd6f Compare August 18, 2026 11:35
@jeqo
jeqo requested a balanced review from Copilot August 18, 2026 11:39
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment on lines +4166 to +4170
val leaderChanged = previousLeaderId.exists(_ != info.partition.leader)
// Skip during controlled shutdown: the leader will not expand ISR for a shutting-down
// broker (isReplicaIsrEligible), and this replica is about to stop serving.
if (seal >= 0 && !isInControlledShutdown &&
(partition.localLogOrException.highWatermark < seal || isOutOfIsr)) {
(partition.localLogOrException.highWatermark < seal || isOutOfIsr || leaderChanged)) {
jeqo added 2 commits August 18, 2026 16:16
maybeShrinkIsr skipped every diskless topic, which includes a partition switched
from classic. Those keep records below classicToDisklessStartOffset in the
replicas' local logs, so a follower that stalls before reaching the seal is
genuinely behind. Nothing removed it from ISR either: the controller shrinks only
on fencing, and the seal is taken at the leader's LEO, so a lagging follower does
not hold the switch back. The replica stayed in ISR without the prefix and was
electable.

Shrink switched partitions, exempting only those with no committed seal:
born-diskless, or a switch aborted through AlterDisklessSwitch. Neither
replicates a classic prefix from this leader, so neither has anything to fall
behind on.

Judge lag against the seal rather than the leader's log end offset. Past the seal
the local log grows only by consolidation, which every replica appends from
object storage through DisklessLeaderEndPoint -- a local fetch that never reaches
the leader. The leader's recorded state for a healthy consolidating follower
therefore stops at the seal while its own LEO keeps advancing, so comparing the
two shrinks the whole ISR out, and nothing re-expands it because expansion also
waits for a follower fetch. Partition takes the ceiling as an optional argument,
defaulted off, so classic partitions keep comparing against the LEO.

Shrinking alone would also regress after a leader election. A new leader resets an
in-ISR follower's recorded state to UNKNOWN (Replica.resetReplicaState), and a
follower at the seal and in ISR starts no fetcher, so the grace period from
lastCaughtUpTimeMs would expire and the follower would be shrunk and re-admitted
on every leader change. Schedule a catch-up fetch when the leader changed, so the
new leader observes the follower's position before the lag timeout. Keyed on the
leader moving rather than on a leader-epoch bump: a bump that keeps the same
leader does not lose the recorded state, and keying on it would also schedule a
pointless fetch on every fresh metadata delta.
The leader-change catch-up test picked broker 3 as the new leader, which
ClusterImageTest.IMAGE1 does n
10BC8
ot register. The leader node therefore never
resolved, applyLocalFollowersDelta called addFetcherForPartitions with an
empty map, and the `times(1)` assertion held on that empty call: it proved
the branch was entered but nothing about where or from which offset the
follower would fetch.

Move the leader from 0 to 2 (both registered in IMAGE1) and assert the
exact InitialFetchState, matching the sibling switch-pending test. Name the
leaders after the production variables the test pins (previousLeaderId /
newLeaderId) and hoist the seal offset into a val.
@jeqo
jeqo force-pushed the jeqo/shrink-switched-isr branch from 8f5cd6f to 90c6a49 Compare August 18, 2026 13:16
…ting followers

Enabling lag-based ISR shrink for switched partitions exposed a second way the
leader loses its record of a healthy follower. A consolidating follower reads the
consolidated suffix from object storage through DisklessLeaderEndPoint, a local
fetch that never reaches the leader, so the leader records that follower only
during the classic phase. makeLeader resets the record to UNKNOWN, which leaves a
new leader with no evidence at all: the lag timeout expires and the ISR shrinks to
the leader alone. Expansion also waits for a follower fetch, so nothing brings the
follower back. Leader changes are routine, so this lands on every rolling restart.

Divert a consolidation-ready follower to the classic ReplicaFetcher for one round
when the leader changed. That fetch records the position on the new leader, then
the fetcher self-evicts at the seal and hands the partition back to consolidation,
which is the handoff a below-seal follower already takes. The diverted partition
also leaves the consolidation fetcher manager: its fetcher from the previous leader
epoch would otherwise keep appending to the same log and trip
processPartitionData's fetchOffset == logEndOffset check.

Cap both sides of the lag comparison. The diversion makes a recorded offset above
the seal reachable, because the leader answers from its local log when its own
consolidation runs ahead of the follower, recording the real offset instead of the
seal. Judging that against a ceiling of the seal fails the offset leg and shrinks
the follower one lag timeout later, so a follower recorded at or beyond the ceiling
now counts as caught up.

Testing: unit tests cover the diversion on a leader change and a follower recorded
beyond the seal staying in ISR. Both fail without the corresponding change. The
upstream PartitionTest suite passes unchanged.
jeqo added a commit that referenced this pull request Aug 18, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Aug 18, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Aug 18, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Aug 18, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Aug 19, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Sep 2, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Sep 7, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Sep 7, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Sep 7, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Sep 8, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Sep 8, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
jeqo added a commit that referenced this pull request Sep 8, 2026
…ranch

The consolidating branch of fetchMessages is not gated on isFromConsumer, so a
follower fetch enters it too. On a consolidating leader the local log end offset
is the consolidated frontier, well past the seal, so a follower fetch at the seal
satisfies fetchOffset < logEndOffset and reads the consolidated suffix from the
leader's local log. That skips the at-seal branch entirely and contradicts the
invariant stated in the same method: followers must never replicate diskless
records into their local log. It also ships those records over the inter-broker
path, billing them to ReplicationBytesInPerSec rather than
ConsolidationFetchBytesInPerSec, which is the traffic consolidation exists to
avoid, and defers ISR admission until the follower matches the frontier.

Exclude a follower fetch at or above a committed seal from the local-log path, so
the at-seal branch answers it with empty records and the high watermark clamped
to the seal.

The exclusion is inert today: a consolidating follower evicts from the classic
fetcher on the same call that brings it to the seal, so it never issues a fetch
at or above the seal. It becomes load-bearing as soon as a consolidating follower
stays on the classic fetcher at the seal, which both the ISR wait in this branch
and the leader-change diversion in #756 introduce.

Testing

testFollowerFetchAtSealOnConsolidatingLeaderGetsAtSealResponse appends past the
seal on the leader, then asserts a follower fetch at the seal receives empty
records with the high watermark at the seal, and that the branch still records
the follower's position. Without the exclusion the follower receives 85 bytes of
the leader's consolidated log.
giuseppelillo added a commit that referenced this pull request Nov 3, 2026
…633]

`maybeShrinkIsr` skipped every diskless topic, including a switched
partition in phase 1. In phase 1 the ISR members' local logs are the only
complete copy of the classic prefix, so a follower that stops fetching
stayed in ISR without it and stayed electable.

The broker now runs the standard `Partition.maybeShrinkIsr` check for a
partition with a committed seal and no `classicPrefixInRemoteStorage` flag.
Born-diskless and phase-2 partitions keep the skip. KC-589 keeps every
phase-1 log at the seal, so the leader's log end offset needs no cap,
unlike in PR #756.

In phase 1, a switched follower leaves the classic fetcher once it's at the
seal and in ISR. A new leader starts with no fetch state for it, so the
shrink would remove it on every leader change. In phase 1 the follower now
schedules one fetch at the seal on a new leader epoch.

## Testing

- New tests in `ReplicaManagerInklessTest`: the shrink gate for phase-1,
  phase-2, and born-diskless partitions, and the seal fetch on a leader
  change in phase 1 but not in phase 2.
- `testApplyDeltaDoesNotStartCatchUpFetcherWhenDisklessFollowerHwmAtSealOffset`,
  `testSwitchedConsolidatingFollowerAtSealHandsOffToConsolidationOnClassicPrefixFlagCommit`,
  and the `DisklessSwitchInvariantsTest` follower scenario at the seal
  reflect the phase-1 seal fetch.
- `ReplicaManagerTest`, `ReplicaFetcherThreadTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation and switch
  suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
giuseppelillo added a commit that referenced this pull request Nov 3, 2026
…633]

`maybeShrinkIsr` skipped every diskless topic, including a switched
partition in phase 1. In phase 1 the ISR members' local logs are the only
complete copy of the classic prefix, so a follower that stops fetching
stayed in ISR without it and stayed electable.

The broker now runs the standard `Partition.maybeShrinkIsr` check for a
partition with a committed seal and no `classicPrefixInRemoteStorage` flag.
Born-diskless and phase-2 partitions keep the skip. KC-589 keeps every
phase-1 log at the seal, so the leader's log end offset needs no cap,
unlike in PR #756.

In phase 1, a switched follower leaves the classic fetcher once it's at the
seal and in ISR. A new leader starts with no fetch state for it, so the
shrink would remove it on every leader change. In phase 1 the follower now
schedules one fetch at the seal on a new leader epoch.

## Testing

- New tests in `ReplicaManagerInklessTest`: the shrink gate for phase-1,
  phase-2, and born-diskless partitions, and the seal fetch on a leader
  change in phase 1 but not in phase 2.
- `testApplyDeltaDoesNotStartCatchUpFetcherWhenDisklessFollowerHwmAtSealOffset`,
  `testSwitchedConsolidatingFollowerAtSealHandsOffToConsolidationOnClassicPrefixFlagCommit`,
  and the `DisklessSwitchInvariantsTest` follower scenario at the seal
  reflect the phase-1 seal fetch.
- `ReplicaManagerTest`, `ReplicaFetcherThreadTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation and switch
  suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
giuseppelillo added a commit that referenced this pull request Nov 4, 2026
…633]

`maybeShrinkIsr` skipped every diskless topic, including a switched
partition in phase 1. In phase 1 the ISR members' local logs are the only
complete copy of the classic prefix, so a follower that stops fetching
stayed in ISR without it and stayed electable.

The broker now runs the standard `Partition.maybeShrinkIsr` check for a
partition with a committed seal and no `classicPrefixInRemoteStorage` flag.
Born-diskless and phase-2 partitions keep the skip. KC-589 keeps every
phase-1 log at the seal, so the leader's log end offset needs no cap,
unlike in PR #756.

In phase 1, a switched follower leaves the classic fetcher once it's at the
seal and in ISR. A new leader starts with no fetch state for it, so the
shrink would remove it on every leader change. In phase 1 the follower now
schedules one fetch at the seal on a new leader epoch.

## Testing

- New tests in `ReplicaManagerInklessTest`: the shrink gate for phase-1,
  phase-2, and born-diskless partitions, and the seal fetch on a leader
  change in phase 1 but not in phase 2.
- `testApplyDeltaDoesNotStartCatchUpFetcherWhenDisklessFollowerHwmAtSealOffset`,
  `testSwitchedConsolidatingFollowerAtSealHandsOffToConsolidationOnClassicPrefixFlagCommit`,
  and the `DisklessSwitchInvariantsTest` follower scenario at the seal
  reflect the phase-1 seal fetch.
- `ReplicaManagerTest`, `ReplicaFetcherThreadTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation and switch
  suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
giuseppelillo added a commit that referenced this pull request Nov 5, 2026
* feat(inkless:switch): gate broker ISR shrink on the switch phase [KC-633]

`maybeShrinkIsr` skipped every diskless topic, including a switched
partition in phase 1. In phase 1 the ISR members' local logs are the only
complete copy of the classic prefix, so a follower that stops fetching
stayed in ISR without it and stayed electable.

The broker now runs the standard `Partition.maybeShrinkIsr` check for a
partition with a committed seal and no `classicPrefixInRemoteStorage` flag.
Born-diskless and phase-2 partitions keep the skip. KC-589 keeps every
phase-1 log at the seal, so the leader's log end offset needs no cap,
unlike in PR #756.

In phase 1, a switched follower leaves the classic fetcher once it's at the
seal and in ISR. A new leader starts with no fetch state for it, so the
shrink would remove it on every leader change. In phase 1 the follower now
schedules one fetch at the seal on a new leader epoch.

## Testing

- New tests in `ReplicaManagerInklessTest`: the shrink gate for phase-1,
  phase-2, and born-diskless partitions, and the seal fetch on a leader
  change in phase 1 but not in phase 2.
- `testApplyDeltaDoesNotStartCatchUpFetcherWhenDisklessFollowerHwmAtSealOffset`,
  `testSwitchedConsolidatingFollowerAtSealHandsOffToConsolidationOnClassicPrefixFlagCommit`,
  and the `DisklessSwitchInvariantsTest` follower scenario at the seal
  reflect the phase-1 seal fetch.
- `ReplicaManagerTest`, `ReplicaFetcherThreadTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation and switch
  suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5

* fix(inkless:switch): skip the phase-1 ISR shrink when the leader log ends past the seal [KC-633]

A leader that consolidated in phase 1 before an upgrade keeps its suffix
above the seal, because the suffix reconciliation checks only whether the
topic consolidates. The leader records a follower fetch at or above the
seal as a fetch at the seal, so lag against its log end offset removes
followers that hold the whole prefix. The leader epoch also starts at the
end of the suffix, so those followers can't rejoin ISR in phase 1.

The phase-1 shrink now runs only when the leader's log ends at or below
the seal. A leader that kept a suffix skips the shrink, as every switched
partition did before this change.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader through `applyDelta` and
  records a follower fetch at the seal. It fails without this change.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `ReplicaFetcherThreadTest`,
  and `DisklessSwitchInvariantsTest` pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): cap the phase-1 ISR checks at the seal [KC-633]

The previous commit skipped the phase-1 shrink when the leader's log ends
past the seal. That leaves a lagging follower in ISR on such a leader. It
also misses a leader whose log ends at the seal but whose leader epoch
starts past it: `makeLeader` sets the epoch start to the log end offset
before `maybeReconcileSwitchedLeader` truncates a tail above the seal. A
follower that the shrink removes from that leader can't rejoin ISR in
phase 1.

In phase 1 the leader records a follower fetch at or above the seal as a
fetch at the seal, so the seal is where follower replication ends.
`ReplicaManager.replicationEndOffset` returns the seal in phase 1, and
`Partition` caps three offsets at it while the partition is sealed: the
log end offset in the lag check, and the high watermark and the leader
epoch start offset in the admission check. A classic partition isn't
sealed, so its checks don't change and don't read metadata.

`Partition.scala` becomes an interleaved path in `INKLESS_OWNERSHIP`.

- `testMaybeShrinkIsrCapsLagAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader. A follower that fetched
  at the seal stays in ISR, and a follower that never fetched leaves it.
- `testFollowerFetchAtSealJoinsIsrWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  checks that a follower at the seal joins ISR when the leader's high
  watermark and leader epoch start offset are past the seal.
- Without the cap, the first test fails for the follower at the seal, and
  the second test fails.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): treat a follower at the old seal as caught up after a re-seal [KC-633]

An operator re-seal lowers the phase-1 seal without changing the leader
epoch. The leader truncates to the new seal and keeps its follower
states, and the follower schedules no new fetch, so a caught-up follower
stays recorded at the old seal. `ReplicaState.isCaughtUp` compares the
recorded offset to the capped leader end offset for equality, so the lag
check removes that follower once the lag time passes, even though it
holds the whole lowered prefix.

While the partition is sealed in phase 1, the lag check now treats a
follower recorded at or past the seal as caught up. The admission check
already accepts that follower.

The previous commit also added a constructor parameter to `Partition`.
Java can't use the Scala default, so the JMH benchmarks that call
`new Partition(...)` stopped compiling, and the `Compile and Check Java`
check failed. The upstream constructor is back, and `Partition.apply`
sets the replication end offset supplier on a field instead.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtOldSealWhenOperatorLowersPhaseOneSeal`
  records a follower at seal 10, lowers the seal to 7 with the same
  leader epoch, and checks that the lag check keeps the follower in ISR.
  Without the follower cap, the test fails with an ISR shrink to the
  leader.
- `./gradlew check -x test -x generateJooqClasses` passes, which is the
  command that `Compile and Check Java` runs.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(inkless:switch): drop the leader cap from the phase-1 lag check [KC-633]

Review follow-ups.

The lag check no longer caps the leader's log end offset at the seal. A
follower recorded at or past the seal is already caught up, and a
follower below the seal can't match the leader's log end offset, capped
or not: a phase-1 leader below the seal goes offline. The cap never
changed the result, so `getOutOfSyncReplicas` keeps the upstream leader
end offset.

`InklessMetadataView.classicPrefixWaitingForUploadSeal` returns the
phase-1 seal from one partition registration read, so the phase and the
seal can't come from two different images. `replicationEndOffset`
becomes `replicationStopOffset`, because it's a ceiling rather than an
end offset next to `logEndOffset`. The Scaladoc drops the upgrade case
and names the operator re-seal, which keeps the leader epoch start above
the lowered seal.

## Testing

- `testFollowerFetchAtLoweredSealJoinsIsrWhenOperatorLowersPhaseOneSeal`
  lowers the seal from 10 to 7 on a leader whose epoch starts at 10, and
  checks that a follower fetching at 7 joins ISR. Without the cap on the
  leader epoch start offset, the test fails.
- `InklessMetadataViewTest` covers `classicPrefixWaitingForUploadSeal`
  for born-diskless, pending, phase-1, phase-2, and missing partitions.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): shrink ISR for lag while the switch is pending [KC-633]

While the classic-to-diskless switch is pending, the leader is sealed, so
its log end offset is frozen. `InitDisklessLogState.WaitingForReplication`
proposes the seal only once the leader's high watermark reaches that log
end offset, and it has no timeout. The previous gate shrank ISR only in
phase 1, so a follower in ISR that stopped fetching below the frozen log
end offset held the high watermark below it and blocked the switch until
the follower came back.

The ISR shrink gate now uses `isClassicPrefixOnlyOnLocalDisks`, which
covers pending and phase 1. The controller uses the same predicate for
unclean election, one-step reassignment, and leader acceptors, and it
accepts `AlterPartition` while the switch is pending. In pending the
leader has no committed seal, so `replicationStopOffset` stays `None` and
the lag check follows classic rules.

## Testing

- `testMaybeShrinkIsrRemovesStalledFollowerBelowFrozenLeoWhileSwitchPending`
  seals a pending leader at log end offset 10 with its high watermark at
  5, and checks that the shrink removes the stalled follower. Without the
  fix, the test fails.
- `testMaybeShrinkIsrGatesDisklessPartitionOnSwitchPhase` gains a
  `PENDING` row that expects a shrink. Without the fix, the row fails.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
gqmelo pushed a commit that referenced this pull request Nov 5, 2026
* feat(inkless:switch): gate broker ISR shrink on the switch phase [KC-633]

`maybeShrinkIsr` skipped every diskless topic, including a switched
partition in phase 1. In phase 1 the ISR members' local logs are the only
complete copy of the classic prefix, so a follower that stops fetching
stayed in ISR without it and stayed electable.

The broker now runs the standard `Partition.maybeShrinkIsr` check for a
partition with a committed seal and no `classicPrefixInRemoteStorage` flag.
Born-diskless and phase-2 partitions keep the skip. KC-589 keeps every
phase-1 log at the seal, so the leader's log end offset needs no cap,
unlike in PR #756.

In phase 1, a switched follower leaves the classic fetcher once it's at the
seal and in ISR. A new leader starts with no fetch state for it, so the
shrink would remove it on every leader change. In phase 1 the follower now
schedules one fetch at the seal on a new leader epoch.

## Testing

- New tests in `ReplicaManagerInklessTest`: the shrink gate for phase-1,
  phase-2, and born-diskless partitions, and the seal fetch on a leader
  change in phase 1 but not in phase 2.
- `testApplyDeltaDoesNotStartCatchUpFetcherWhenDisklessFollowerHwmAtSealOffset`,
  `testSwitchedConsolidatingFollowerAtSealHandsOffToConsolidationOnClassicPrefixFlagCommit`,
  and the `DisklessSwitchInvariantsTest` follower scenario at the seal
  reflect the phase-1 seal fetch.
- `ReplicaManagerTest`, `ReplicaFetcherThreadTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation and switch
  suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5

* fix(inkless:switch): skip the phase-1 ISR shrink when the leader log ends past the seal [KC-633]

A leader that consolidated in phase 1 before an upgrade keeps its suffix
above the seal, because the suffix reconciliation checks only whether the
topic consolidates. The leader records a follower fetch at or above the
seal as a fetch at the seal, so lag against its log end offset removes
followers that hold the whole prefix. The leader epoch also starts at the
end of the suffix, so those followers can't rejoin ISR in phase 1.

The phase-1 shrink now runs only when the leader's log ends at or below
the seal. A leader that kept a suffix skips the shrink, as every switched
partition did before this change.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader through `applyDelta` and
  records a follower fetch at the seal. It fails without this change.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `ReplicaFetcherThreadTest`,
  and `DisklessSwitchInvariantsTest` pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): cap the phase-1 ISR checks at the seal [KC-633]

The previous commit skipped the phase-1 shrink when the leader's log ends
past the seal. That leaves a lagging follower in ISR on such a leader. It
also misses a leader whose log ends at the seal but whose leader epoch
starts past it: `makeLeader` sets the epoch start to the log end offset
before `maybeReconcileSwitchedLeader` truncates a tail above the seal. A
follower that the shrink removes from that leader can't rejoin ISR in
phase 1.

In phase 1 the leader records a follower fetch at or above the seal as a
fetch at the seal, so the seal is where follower replication ends.
`ReplicaManager.replicationEndOffset` returns the seal in phase 1, and
`Partition` caps three offsets at it while the partition is sealed: the
log end offset in the lag check, and the high watermark and the leader
epoch start offset in the admission check. A classic partition isn't
sealed, so its checks don't change and don't read metadata.

`Partition.scala` becomes an interleaved path in `INKLESS_OWNERSHIP`.

- `testMaybeShrinkIsrCapsLagAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader. A follower that fetched
  at the seal stays in ISR, and a follower that never fetched leaves it.
- `testFollowerFetchAtSealJoinsIsrWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  checks that a follower at the seal joins ISR when the leader's high
  watermark and leader epoch start offset are past the seal.
- Without the cap, the first test fails for the follower at the seal, and
  the second test fails.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): treat a follower at the old seal as caught up after a re-seal [KC-633]

An operator re-seal lowers the phase-1 seal without changing the leader
epoch. The leader truncates to the new seal and keeps its follower
states, and the follower schedules no new fetch, so a caught-up follower
stays recorded at the old seal. `ReplicaState.isCaughtUp` compares the
recorded offset to the capped leader end offset for equality, so the lag
check removes that follower once the lag time passes, even though it
holds the whole lowered prefix.

While the partition is sealed in phase 1, the lag check now treats a
follower recorded at or past the seal as caught up. The admission check
already accepts that follower.

The previous commit also added a constructor parameter to `Partition`.
Java can't use the Scala default, so the JMH benchmarks that call
`new Partition(...)` stopped compiling, and the `Compile and Check Java`
check failed. The upstream constructor is back, and `Partition.apply`
sets the replication end offset supplier on a field instead.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtOldSealWhenOperatorLowersPhaseOneSeal`
  records a follower at seal 10, lowers the seal to 7 with the same
  leader epoch, and checks that the lag check keeps the follower in ISR.
  Without the follower cap, the test fails with an ISR shrink to the
  leader.
- `./gradlew check -x test -x generateJooqClasses` passes, which is the
  command that `Compile and Check Java` runs.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(inkless:switch): drop the leader cap from the phase-1 lag check [KC-633]

Review follow-ups.

The lag check no longer caps the leader's log end offset at the seal. A
follower recorded at or past the seal is already caught up, and a
follower below the seal can't match the leader's log end offset, capped
or not: a phase-1 leader below the seal goes offline. The cap never
changed the result, so `getOutOfSyncReplicas` keeps the upstream leader
end offset.

`InklessMetadataView.classicPrefixWaitingForUploadSeal` returns the
phase-1 seal from one partition registration read, so the phase and the
seal can't come from two different images. `replicationEndOffset`
becomes `replicationStopOffset`, because it's a ceiling rather than an
end offset next to `logEndOffset`. The Scaladoc drops the upgrade case
and names the operator re-seal, which keeps the leader epoch start above
the lowered seal.

## Testing

- `testFollowerFetchAtLoweredSealJoinsIsrWhenOperatorLowersPhaseOneSeal`
  lowers the seal from 10 to 7 on a leader whose epoch starts at 10, and
  checks that a follower fetching at 7 joins ISR. Without the cap on the
  leader epoch start offset, the test fails.
- `InklessMetadataViewTest` covers `classicPrefixWaitingForUploadSeal`
  for born-diskless, pending, phase-1, phase-2, and missing partitions.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): shrink ISR for lag while the switch is pending [KC-633]

While the classic-to-diskless switch is pending, the leader is sealed, so
its log end offset is frozen. `InitDisklessLogState.WaitingForReplication`
proposes the seal only once the leader's high watermark reaches that log
end offset, and it has no timeout. The previous gate shrank ISR only in
phase 1, so a follower in ISR that stopped fetching below the frozen log
end offset held the high watermark below it and blocked the switch until
the follower came back.

The ISR shrink gate now uses `isClassicPrefixOnlyOnLocalDisks`, which
covers pending and phase 1. The controller uses the same predicate for
unclean election, one-step reassignment, and leader acceptors, and it
accepts `AlterPartition` while the switch is pending. In pending the
leader has no committed seal, so `replicationStopOffset` stays `None` and
the lag check follows classic rules.

## Testing

- `testMaybeShrinkIsrRemovesStalledFollowerBelowFrozenLeoWhileSwitchPending`
  seals a pending leader at log end offset 10 with its high watermark at
  5, and checks that the shrink removes the stalled follower. Without the
  fix, the test fails.
- `testMaybeShrinkIsrGatesDisklessPartitionOnSwitchPhase` gains a
  `PENDING` row that expects a shrink. Without the fix, the row fails.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
gqmelo pushed a commit that referenced this pull request Nov 5, 2026
* feat(inkless:switch): gate broker ISR shrink on the switch phase [KC-633]

`maybeShrinkIsr` skipped every diskless topic, including a switched
partition in phase 1. In phase 1 the ISR members' local logs are the only
complete copy of the classic prefix, so a follower that stops fetching
stayed in ISR without it and stayed electable.

The broker now runs the standard `Partition.maybeShrinkIsr` check for a
partition with a committed seal and no `classicPrefixInRemoteStorage` flag.
Born-diskless and phase-2 partitions keep the skip. KC-589 keeps every
phase-1 log at the seal, so the leader's log end offset needs no cap,
unlike in PR #756.

In phase 1, a switched follower leaves the classic fetcher once it's at the
seal and in ISR. A new leader starts with no fetch state for it, so the
shrink would remove it on every leader change. In phase 1 the follower now
schedules one fetch at the seal on a new leader epoch.

## Testing

- New tests in `ReplicaManagerInklessTest`: the shrink gate for phase-1,
  phase-2, and born-diskless partitions, and the seal fetch on a leader
  change in phase 1 but not in phase 2.
- `testApplyDeltaDoesNotStartCatchUpFetcherWhenDisklessFollowerHwmAtSealOffset`,
  `testSwitchedConsolidatingFollowerAtSealHandsOffToConsolidationOnClassicPrefixFlagCommit`,
  and the `DisklessSwitchInvariantsTest` follower scenario at the seal
  reflect the phase-1 seal fetch.
- `ReplicaManagerTest`, `ReplicaFetcherThreadTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation and switch
  suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5

* fix(inkless:switch): skip the phase-1 ISR shrink when the leader log ends past the seal [KC-633]

A leader that consolidated in phase 1 before an upgrade keeps its suffix
above the seal, because the suffix reconciliation checks only whether the
topic consolidates. The leader records a follower fetch at or above the
seal as a fetch at the seal, so lag against its log end offset removes
followers that hold the whole prefix. The leader epoch also starts at the
end of the suffix, so those followers can't rejoin ISR in phase 1.

The phase-1 shrink now runs only when the leader's log ends at or below
the seal. A leader that kept a suffix skips the shrink, as every switched
partition did before this change.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader through `applyDelta` and
  records a follower fetch at the seal. It fails without this change.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `ReplicaFetcherThreadTest`,
  and `DisklessSwitchInvariantsTest` pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): cap the phase-1 ISR checks at the seal [KC-633]

The previous commit skipped the phase-1 shrink when the leader's log ends
past the seal. That leaves a lagging follower in ISR on such a leader. It
also misses a leader whose log ends at the seal but whose leader epoch
starts past it: `makeLeader` sets the epoch start to the log end offset
before `maybeReconcileSwitchedLeader` truncates a tail above the seal. A
follower that the shrink removes from that leader can't rejoin ISR in
phase 1.

In phase 1 the leader records a follower fetch at or above the seal as a
fetch at the seal, so the seal is where follower replication ends.
`ReplicaManager.replicationEndOffset` returns the seal in phase 1, and
`Partition` caps three offsets at it while the partition is sealed: the
log end offset in the lag check, and the high watermark and the leader
epoch start offset in the admission check. A classic partition isn't
sealed, so its checks don't change and don't read metadata.

`Partition.scala` becomes an interleaved path in `INKLESS_OWNERSHIP`.

- `testMaybeShrinkIsrCapsLagAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader. A follower that fetched
  at the seal stays in ISR, and a follower that never fetched leaves it.
- `testFollowerFetchAtSealJoinsIsrWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  checks that a follower at the seal joins ISR when the leader's high
  watermark and leader epoch start offset are past the seal.
- Without the cap, the first test fails for the follower at the seal, and
  the second test fails.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): treat a follower at the old seal as caught up after a re-seal [KC-633]

An operator re-seal lowers the phase-1 seal without changing the leader
epoch. The leader truncates to the new seal and keeps its follower
states, and the follower schedules no new fetch, so a caught-up follower
stays recorded at the old seal. `ReplicaState.isCaughtUp` compares the
recorded offset to the capped leader end offset for equality, so the lag
check removes that follower once the lag time passes, even though it
holds the whole lowered prefix.

While the partition is sealed in phase 1, the lag check now treats a
follower recorded at or past the seal as caught up. The admission check
already accepts that follower.

The previous commit also added a constructor parameter to `Partition`.
Java can't use the Scala default, so the JMH benchmarks that call
`new Partition(...)` stopped compiling, and the `Compile and Check Java`
check failed. The upstream constructor is back, and `Partition.apply`
sets the replication end offset supplier on a field instead.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtOldSealWhenOperatorLowersPhaseOneSeal`
  records a follower at seal 10, lowers the seal to 7 with the same
  leader epoch, and checks that the lag check keeps the follower in ISR.
  Without the follower cap, the test fails with an ISR shrink to the
  leader.
- `./gradlew check -x test -x generateJooqClasses` passes, which is the
  command that `Compile and Check Java` runs.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(inkless:switch): drop the leader cap from the phase-1 lag check [KC-633]

Review follow-ups.

The lag check no longer caps the leader's log end offset at the seal. A
follower recorded at or past the seal is already caught up, and a
follower below the seal can't match the leader's log end offset, capped
or not: a phase-1 leader below the seal goes offline. The cap never
changed the result, so `getOutOfSyncReplicas` keeps the upstream leader
end offset.

`InklessMetadataView.classicPrefixWaitingForUploadSeal` returns the
phase-1 seal from one partition registration read, so the phase and the
seal can't come from two different images. `replicationEndOffset`
becomes `replicationStopOffset`, because it's a ceiling rather than an
end offset next to `logEndOffset`. The Scaladoc drops the upgrade case
and names the operator re-seal, which keeps the leader epoch start above
the lowered seal.

## Testing

- `testFollowerFetchAtLoweredSealJoinsIsrWhenOperatorLowersPhaseOneSeal`
  lowers the seal from 10 to 7 on a leader whose epoch starts at 10, and
  checks that a follower fetching at 7 joins ISR. Without the cap on the
  leader epoch start offset, the test fails.
- `InklessMetadataViewTest` covers `classicPrefixWaitingForUploadSeal`
  for born-diskless, pending, phase-1, phase-2, and missing partitions.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): shrink ISR for lag while the switch is pending [KC-633]

While the classic-to-diskless switch is pending, the leader is sealed, so
its log end offset is frozen. `InitDisklessLogState.WaitingForReplication`
proposes the seal only once the leader's high watermark reaches that log
end offset, and it has no timeout. The previous gate shrank ISR only in
phase 1, so a follower in ISR that stopped fetching below the frozen log
end offset held the high watermark below it and blocked the switch until
the follower came back.

The ISR shrink gate now uses `isClassicPrefixOnlyOnLocalDisks`, which
covers pending and phase 1. The controller uses the same predicate for
unclean election, one-step reassignment, and leader acceptors, and it
accepts `AlterPartition` while the switch is pending. In pending the
leader has no committed seal, so `replicationStopOffset` stays `None` and
the lag check follows classic rules.

## Testing

- `testMaybeShrinkIsrRemovesStalledFollowerBelowFrozenLeoWhileSwitchPending`
  seals a pending leader at log end offset 10 with its high watermark at
  5, and checks that the shrink removes the stalled follower. Without the
  fix, the test fails.
- `testMaybeShrinkIsrGatesDisklessPartitionOnSwitchPhase` gains a
  `PENDING` row that expects a shrink. Without the fix, the row fails.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
gqmelo pushed a commit that referenced this pull request Nov 5, 2026
* feat(inkless:switch): gate broker ISR shrink on the switch phase [KC-633]

`maybeShrinkIsr` skipped every diskless topic, including a switched
partition in phase 1. In phase 1 the ISR members' local logs are the only
complete copy of the classic prefix, so a follower that stops fetching
stayed in ISR without it and stayed electable.

The broker now runs the standard `Partition.maybeShrinkIsr` check for a
partition with a committed seal and no `classicPrefixInRemoteStorage` flag.
Born-diskless and phase-2 partitions keep the skip. KC-589 keeps every
phase-1 log at the seal, so the leader's log end offset needs no cap,
unlike in PR #756.

In phase 1, a switched follower leaves the classic fetcher once it's at the
seal and in ISR. A new leader starts with no fetch state for it, so the
shrink would remove it on every leader change. In phase 1 the follower now
schedules one fetch at the seal on a new leader epoch.

## Testing

- New tests in `ReplicaManagerInklessTest`: the shrink gate for phase-1,
  phase-2, and born-diskless partitions, and the seal fetch on a leader
  change in phase 1 but not in phase 2.
- `testApplyDeltaDoesNotStartCatchUpFetcherWhenDisklessFollowerHwmAtSealOffset`,
  `testSwitchedConsolidatingFollowerAtSealHandsOffToConsolidationOnClassicPrefixFlagCommit`,
  and the `DisklessSwitchInvariantsTest` follower scenario at the seal
  reflect the phase-1 seal fetch.
- `ReplicaManagerTest`, `ReplicaFetcherThreadTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation and switch
  suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5

* fix(inkless:switch): skip the phase-1 ISR shrink when the leader log ends past the seal [KC-633]

A leader that consolidated in phase 1 before an upgrade keeps its suffix
above the seal, because the suffix reconciliation checks only whether the
topic consolidates. The leader records a follower fetch at or above the
seal as a fetch at the seal, so lag against its log end offset removes
followers that hold the whole prefix. The leader epoch also starts at the
end of the suffix, so those followers can't rejoin ISR in phase 1.

The phase-1 shrink now runs only when the leader's log ends at or below
the seal. A leader that kept a suffix skips the shrink, as every switched
partition did before this change.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader through `applyDelta` and
  records a follower fetch at the seal. It fails without this change.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `ReplicaFetcherThreadTest`,
  and `DisklessSwitchInvariantsTest` pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): cap the phase-1 ISR checks at the seal [KC-633]

The previous commit skipped the phase-1 shrink when the leader's log ends
past the seal. That leaves a lagging follower in ISR on such a leader. It
also misses a leader whose log ends at the seal but whose leader epoch
starts past it: `makeLeader` sets the epoch start to the log end offset
before `maybeReconcileSwitchedLeader` truncates a tail above the seal. A
follower that the shrink removes from that leader can't rejoin ISR in
phase 1.

In phase 1 the leader records a follower fetch at or above the seal as a
fetch at the seal, so the seal is where follower replication ends.
`ReplicaManager.replicationEndOffset` returns the seal in phase 1, and
`Partition` caps three offsets at it while the partition is sealed: the
log end offset in the lag check, and the high watermark and the leader
epoch start offset in the admission check. A classic partition isn't
sealed, so its checks don't change and don't read metadata.

`Partition.scala` becomes an interleaved path in `INKLESS_OWNERSHIP`.

- `testMaybeShrinkIsrCapsLagAtSealWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  keeps a consolidated suffix on a phase-1 leader. A follower that fetched
  at the seal stays in ISR, and a follower that never fetched leaves it.
- `testFollowerFetchAtSealJoinsIsrWhenPhaseOneLeaderKeepsConsolidatedSuffix`
  checks that a follower at the seal joins ISR when the leader's high
  watermark and leader epoch start offset are past the seal.
- Without the cap, the first test fails for the follower at the seal, and
  the second test fails.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): treat a follower at the old seal as caught up after a re-seal [KC-633]

An operator re-seal lowers the phase-1 seal without changing the leader
epoch. The leader truncates to the new seal and keeps its follower
states, and the follower schedules no new fetch, so a caught-up follower
stays recorded at the old seal. `ReplicaState.isCaughtUp` compares the
recorded offset to the capped leader end offset for equality, so the lag
check removes that follower once the lag time passes, even though it
holds the whole lowered prefix.

While the partition is sealed in phase 1, the lag check now treats a
follower recorded at or past the seal as caught up. The admission check
already accepts that follower.

The previous commit also added a constructor parameter to `Partition`.
Java can't use the Scala default, so the JMH benchmarks that call
`new Partition(...)` stopped compiling, and the `Compile and Check Java`
check failed. The upstream constructor is back, and `Partition.apply`
sets the replication end offset supplier on a field instead.

## Testing

- `testMaybeShrinkIsrKeepsFollowerAtOldSealWhenOperatorLowersPhaseOneSeal`
  records a follower at seal 10, lowers the seal to 7 with the same
  leader epoch, and checks that the lag check keeps the follower in ISR.
  Without the follower cap, the test fails with an ISR shrink to the
  leader.
- `./gradlew check -x test -x generateJooqClasses` passes, which is the
  command that `Compile and Check Java` runs.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InitDisklessLogManagerTest`, `InklessTopicTypeSwitcherClusterTest`, and
  the consolidation suites pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(inkless:switch): drop the leader cap from the phase-1 lag check [KC-633]

Review follow-ups.

The lag check no longer caps the leader's log end offset at the seal. A
follower recorded at or past the seal is already caught up, and a
follower below the seal can't match the leader's log end offset, capped
or not: a phase-1 leader below the seal goes offline. The cap never
changed the result, so `getOutOfSyncReplicas` keeps the upstream leader
end offset.

`InklessMetadataView.classicPrefixWaitingForUploadSeal` returns the
phase-1 seal from one partition registration read, so the phase and the
seal can't come from two different images. `replicationEndOffset`
becomes `replicationStopOffset`, because it's a ceiling rather than an
end offset next to `logEndOffset`. The Scaladoc drops the upgrade case
and names the operator re-seal, which keeps the leader epoch start above
the lowered seal.

## Testing

- `testFollowerFetchAtLoweredSealJoinsIsrWhenOperatorLowersPhaseOneSeal`
  lowers the seal from 10 to 7 on a leader whose epoch starts at 10, and
  checks that a follower fetching at 7 joins ISR. Without the cap on the
  leader epoch start offset, the test fails.
- `InklessMetadataViewTest` covers `classicPrefixWaitingForUploadSeal`
  for born-diskless, pending, phase-1, phase-2, and missing partitions.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): shrink ISR for lag while the switch is pending [KC-633]

While the classic-to-diskless switch is pending, the leader is sealed, so
its log end offset is frozen. `InitDisklessLogState.WaitingForReplication`
proposes the seal only once the leader's high watermark reaches that log
end offset, and it has no timeout. The previous gate shrank ISR only in
phase 1, so a follower in ISR that stopped fetching below the frozen log
end offset held the high watermark below it and blocked the switch until
the follower came back.

The ISR shrink gate now uses `isClassicPrefixOnlyOnLocalDisks`, which
covers pending and phase 1. The controller uses the same predicate for
unclean election, one-step reassignment, and leader acceptors, and it
accepts `AlterPartition` while the switch is pending. In pending the
leader has no committed seal, so `replicationStopOffset` stays `None` and
the lag check follows classic rules.

## Testing

- `testMaybeShrinkIsrRemovesStalledFollowerBelowFrozenLeoWhileSwitchPending`
  seals a pending leader at log end offset 10 with its high watermark at
  5, and checks that the shrink removes the stalled follower. Without the
  fix, the test fails.
- `testMaybeShrinkIsrGatesDisklessPartitionOnSwitchPhase` gains a
  `PENDING` row that expects a shrink. Without the fix, the row fails.
- `./gradlew check -x test -x generateJooqClasses` passes.
- `ReplicaManagerInklessTest`, `ReplicaManagerTest`, `PartitionTest`,
  `PartitionLockTest`, `IsrExpirationTest`, `ReplicaFetcherThreadTest`,
  `DisklessSwitchInvariantsTest`, `DisklessSwitchFlowTest`,
  `InklessMetadataViewTest`, `InitDisklessLogManagerTest`,
  `InklessTopicTypeSwitcherClusterTest`, and the consolidation suites
  pass.
- `make vale` reports no alerts.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0