jeqo · Nov 6, 2026 · Nov 6, 2026
diff --git a/docs/inkless/CHANGELOG.md b/docs/inkless/CHANGELOG.md
@@ -41,6 +41,67 @@ Regenerate a draft with the `inkless-changelog` skill (or run it directly):
 
 ---
 
+## 0.51 (Kafka 4.1.2, 4.2.2, 4.3.1)
+
+### Features
+- (inkless:switch) gate broker ISR shrink on the switch phase (#841)
+- (inkless:switch) gate consolidation on the switch phase (#840)
+- (inkless:consolidation) bound the WAL prune per request [KC-444] (#843)
+- (inkless:switch) detect classic-prefix upload and request the tiered flag [KC-588] (#835)
+- (inkless:controller) gate ISR, reassignment, and leader election on switch phase (#833)
+- (inkless:sync) label upstream sync PRs with `upstream` (#829)
+
+### Fixes
+- (inkless:controller) skip the config record when the connection string of the control plane doesn't change (#852)
+- (inkless:controller) block unclean leader election in switch phase 1 [KC-636] (#849)
+- (inkless:consolidation) defer prefix reads on a rebuilding replica [KC-673] (#847)
+- (inkless:controller) validate remote storage on consolidated diskless creates (#845)
+- (inkless:metadata) route consolidating partitions with one replica to the replica [KC-651] (#844)
+- (inkless:controller) parse `diskless.enable` case-insensitively on topic creation (#846)
+- (inkless:controller) return a retriable error when the control plane is unavailable [KC-649] (#839)
+- (inkless:control-plane) read unavailableReason once in recordGatedCall [KC-624] (#836)
+- (inkless:control-plane) close the retired delegate before building its replacement [KC-630] (#838)
+- (inkless:controller) skip control plane calls for classic-only topic requests [KC-621] (#832)
+- (inkless:ai) update vale config (#823)
+
+### Tests
+- (inkless:switch) run the stale high watermark promotion test in phase 2 [KC-721] (#854)
+- (inkless:control-plane) cover the availability gate across outages, restarts, and migration locks [KC-619] (#850)
+- (inkless) wait for the published first attempt in reconciler tests [KC-631] (#837)
+- (inkless:switch) check EARLIEST and remote expiration before the first prune [KC-581] (#831)
+
+### Docs
+- (inkless:ai) ask for imports over fully qualified class names (#842)
+- (inkless) document and test the migration of untiered diskless topics (#834)
+- (inkless:release) add 0.50 changelog and cherry-pick sessions (#830)
+
+### Config & metric changes
+- config added: `inkless.consolidation.cleanup.max.batches.per.request` (default 2000). It caps the batches that one consolidation cleanup request deletes per partition. A value of 0 removes the cap. With the default interval, the cap sustains about 33 batches per second per partition, and a backlog of 1,000,000 batches drains in about 8 hours (#843).
+- config changed: the default of `inkless.consolidation.cleanup.interval.ms` drops from 300000 (5 minutes) to 60000 (1 minute), which matches `retention.enforcement.interval.ms` (#843).
+- metric mbean added: `io.aiven.inkless.delete:type=ConsolidatedDisklessLogPruner`, attributes `ConsolidationPruneTotalBatchesDeleted`, `ConsolidationPruneTotalBytesDeleted`, and `ConsolidationPruneTotalTime`. The totals update only after a run completes. `ConsolidationPruneErrorRate` counts runs that fail with an exception (#843).
+- metric added: `kafka.server:type=ReplicaManager,name=ConsolidationDeferredPrefixFetchRate`. It counts consumer fetches for which the broker returns `OFFSET_NOT_AVAILABLE` because its replica is rebuilding the tier state and can't read the prefix yet (#847).
+- metric added: the per-partition gauge `ConsolidationTierRebuildRequests` counts each `OFFSET_MOVED_TO_TIERED_STORAGE` answer that starts a tier-state rebuild on the replica's broker. A retried rebuild counts again (#847).
+- metric added: `kafka.controller:type=KafkaController,name=DisklessSwitchedPartitionsPendingUploadCount`. It counts switched partitions in phase 1, whose classic prefix isn't in remote storage yet (#835).
+- `metrics.rst` now lists the `PruneDisklessLogs` attributes of `io.aiven.inkless.control_plane.postgres:type=PostgresControlPlane` (`PruneDisklessLogsQueryRate`, `PruneDisklessLogsQueryTime`, and `PruneDisklessLogsLastSuccessfulQueryAgeMs`). The broker already registered them, so only the documentation is new (#843).
+
+### Postgres schema changes
+- `V31__Bound_prune_batches.sql`: `CREATE TYPE prune_batches_below_highest_tiered_offset_response_v2` and `CREATE FUNCTION prune_batches_below_highest_tiered_offset_v2`. The migration adds a type and a function, takes no table lock, and doesn't need a `CONCURRENTLY` step. The v1 function stays, so brokers on 0.50 keep pruning during a rolling upgrade (#843).
+
+### Upgrade notes
+- The switch now has two phases. Phase 1 starts when the seal commits. In phase 1, the classic prefix exists only on the local logs of the ISR members, so the broker and controller keep the classic rules for that prefix:
+  - Brokers shrink the ISR (#841).
+  - Brokers don't consolidate the partition (#840).
+  - The controller stages reassignments and requires an online log directory to elect a leader (#833).
+  - The controller rejects unclean leader elections, including a cluster-level `unclean.leader.election.enable=true` (#849).
+
+  Phase 2 starts when the leader confirms the prefix is in remote storage and the controller commits `classicPrefixInRemoteStorage` (#835). For more information, see "Switch phases" in [CLASSIC_TO_DISKLESS_SWITCH.md](CLASSIC_TO_DISKLESS_SWITCH.md).
+- The controller rejects any change to the seal after it commits `classicPrefixInRemoteStorage`. `kafka-topic-switch.sh seal` reports that reason, including with `--dry-run` (#833).
+- CreateTopics, DeleteTopics, and CreatePartitions return the retriable `KAFKA_STORAGE_ERROR` for the affected diskless topics when the control plane is unavailable. Earlier releases returned `UNKNOWN_SERVER_ERROR` (#839).
+- With consolidation on, the controller validates new diskless topics as tiered topics. A create whose retention is below the broker `log.local.retention.ms` now fails at create time. Before, the create succeeded and every later AlterConfigs failed with `INVALID_CONFIG` (#845).
+- `inkless-4.1` and `inkless-4.2` keep the four new `forceDeleteLocalSegmentsBelow` tests in the Scala `kafka.log.UnifiedLogTest`, because their Java `UnifiedLogTest` is a stub (#840). On `inkless-4.1`, the SpotBugs exclude filter skips `SKIPPED_CLASS_TOO_BIG` for `kafka.server.ReplicaManager`, which grows past the analysis limit with #847. `inkless-4.1` also drops a stale `testcontainers` 1.20.3 entry that made its Testcontainers integration tests fail on the `ubuntu-26.04` runners. None of these build changes affects broker behavior.
+
+---
+
 ## 0.50 (Kafka 4.1.2, 4.2.2, 4.3.1)
 
 ### Features

@@ -0,0 +1,96 @@
+# Cherry-pick session: `inkless-4.1`
+
+## Session info
+- **Date**: 2026-11-05
+- **Source**: origin/main (6074c0650f, #854)
+- **Target**: inkless-4.1
+- **Base**: inkless-4.1.2-0.50
+- **Status**: complete, pushed, released in `inkless-release-0.51`
+
+---
+
+## Commits to cherry-pick
+
+| # | Hash | PR | Subject | Status |
+|---|------|-----|---------|--------|
+| 1 | 7a41a38ec9 | #823 | `fix(inkless:ai): update vale config` | Applied |
+| 2 | 08e6a71734 | #831 | `test(inkless:switch): check EARLIEST and remote expiration before the first prune [KC-581]` | Applied |
+| 3 | 4fc13dbaba | #832 | `fix(inkless:controller): skip control plane calls for classic-only topic requests [KC-621]` | Applied |
+| 4 | b78206f6af | #830 | `docs(inkless:release): add 0.50 changelog and cherry-pick sessions` | Applied |
+| 5 | 6c6483b729 | #834 | `docs(inkless): document and test the migration of untiered diskless topics` | Applied |
+| 6 | 27b691d570 | #833 | `feat(inkless:controller): gate ISR, reassignment, and leader election on switch phase` | Applied |
+| 7 | be0ef1b9fd | #837 | `test(inkless): wait for the published first attempt in reconciler tests [KC-631]` | Applied |
+| 8 | 7bb233d5ac | #838 | `fix(inkless:control-plane): close the retired delegate before building its replacement [KC-630]` | Applied |
+| 9 | d7b603ea2a | #836 | `fix(inkless:control-plane): read unavailableReason once in recordGatedCall [KC-624]` | Applied |
+| 10 | 6ef58bd1e5 | #835 | `feat(inkless:switch): detect classic-prefix upload and request the tiered flag [KC-588]` | Applied (conflict) |
+| 11 | 13fb9116cb | #842 | `docs(inkless:ai): ask for imports over fully qualified class names` | Applied |
+| 12 | 63cc8f0814 | #839 | `fix(inkless:controller): return a retriable error when the control plane is unavailable [KC-649]` | Applied |
+| 13 | 36d39041aa | #846 | `fix(inkless:controller): parse diskless.enable case-insensitively on topic creation` | Applied |
+| 14 | 1af55a0a82 | #843 | `feat(inkless:consolidation): bound the WAL prune per request [KC-444]` | Applied |
+| 15 | eea3c262f8 | #844 | `fix(inkless:metadata): route consolidating partitions with one replica to the replica [KC-651]` | Applied |
+| 16 | c8bb204a74 | #845 | `fix(inkless:controller): validate remote storage on consolidated diskless creates` | Applied |
+| 17 | 9bc310258d | #840 | `feat(inkless:switch): gate consolidation on the switch phase` | Applied (conflict) |
+| 18 | f30bf465a1 | #847 | `fix(inkless:consolidation): defer prefix reads on a rebuilding replica [KC-673]` | Applied (conflict) |
+| 19 | 84be446fc0 | #850 | `test(inkless:control-plane): cover the availability gate across outages, restarts, and migration locks [KC-619]` | Applied |
+| 20 | 5851744471 | #849 | `fix(inkless:controller): block unclean leader election in switch phase 1 [KC-636]` | Applied (conflict) |
+| 21 | e2921f6265 | #852 | `fix(inkless:controller): skip unchanged control plane connection string record` | Applied (conflict) |
+| 22 | 9957f7f383 | #841 | `feat(inkless:switch): gate broker ISR shrink on the switch phase` | Applied |
+| 23 | 6074c0650f | #854 | `test(inkless:switch): run the stale high watermark promotion test in phase 2 [KC-721]` | Applied |
+
+---
+
+## Conflict resolution log
+
+### #835 - classic-prefix upload and the tiered flag
+
+**File**: `core/src/test/scala/unit/kafka/server/metadata/DisklessSwitchFlowTest.scala`
+- **Conflict**: the `kafka.server` import. The release branch also imports `HostedPartition`.
+- **Resolution**: kept `HostedPartition` and added `SendingFlagToController`.
+
+### #840 - gate consolidation on the switch phase
+
+**Files**: `core/.../ReplicaAlterLogDirsThreadTest.scala`, `storage/.../UnifiedLogTest.java`
+- **Conflict**: `MemoryRecords` is in `org.apache.kafka.common.record`. The Java `UnifiedLogTest` is a stub, and the conflict pulled in about 6,700 lines of upstream tests.
+- **Resolution**: imported `MemoryRecords` and `SimpleRecord` from `org.apache.kafka.common.record`. Kept the Java stub and ported the four `forceDeleteLocalSegmentsBelow` tests to the Scala `kafka.log.UnifiedLogTest`.
+
+### #847 - deferred prefix reads
+
+**Files**: `core/.../ReplicaManager.scala`, `core/.../DelayedFetch.scala`, `core/.../ReplicaManagerInklessTest.scala`
+- **Conflict**: imports. The release branch also imports `KRaftMetadataCache`.
+- **Resolution**: took the incoming `ReplicaManager` import list and kept `KRaftMetadataCache`. In `DelayedFetch`, kept 4.1's Scala `map` over the status map and passed `deferPrefixReads = true`. In `ReplicaManagerInklessTest`, added `RemoteLogReadResult` and `RemoteStorageFetchInfo` to 4.1's import list, which has no `LogReadResult` there.
+
+### #849 - unclean leader election in switch phase 1
+
+**File**: `metadata/.../ReplicationControlManager.java`
+- **Conflict**: `maybeTriggerUncleanLeaderElectionForLeaderlessPartitions` has no `partitionId` local on the release branch.
+- **Resolution**: removed the pending-switch check and added the local classic prefix check inside the unclean branch, with `topicIdPartition.partitionId()`.
+
+### #852 - skip the record for an unchanged connection string
+
+**Files**: `ConfigurationControlManager.java`, `ConfigurationControlManagerTest.java`
+- **Conflict**: the incoming side adds imports and a cordoned log dirs test that the release branch lacks.
+- **Resolution**: added only the `InklessControlPlaneConfigs` imports and the new tests. Dropped the upstream `testCordonedLogDirsFeature` from the incoming side.
+
+---
+
+## Compilation errors
+
+Same fixes as 4.2 (errors 2 to 4), plus:
+
+| # | File | Error | Cherry-pick | Fix | Status |
+|---|------|-------|-------------|-----|--------|
+| 5 | `core/.../ReplicaManager.scala` | `RemoteStorageFetchInfo` has no `topicIdPartition` | #847 | `topicIdPartition(remoteFetchInfo.topicPartition)` | Fixed |
+| 6 | `core/.../ReplicaManagerInklessTest.scala` | `RemoteStorageFetchInfo` takes a `TopicPartition`; `LogReadResult` takes no `Errors` | #847 | `tp.topicPartition()`; dropped `Errors.NONE` | Fixed |
+| 7 | `storage/.../RemoteLogManagerTest.java` | `RetriableRemoteStorageException` doesn't exist | #831 | assert `RemoteStorageException`, which 4.1 throws | Fixed |
+| 8 | `storage/.../RemoteLogManagerTest.java` | `RemoteStorageFetchInfo` takes a `TopicPartition` | #847 | pass `tp` | Fixed |
+| 9 | `InklessTopicTypeSwitcherClusterTest.java`, `InklessUntieredDisklessMigrationTest.java` | `REMOTE_LOG_MANAGER_FOLLOWER_THREAD_POOL_SIZE_PROP` doesn't exist | #834, #840 | `REMOTE_LOG_MANAGER_THREAD_POOL_SIZE_PROP`, which sizes the follower pool on 4.1 | Fixed |
+
+Error 5 is the only change to main code. All fixes are in one sync(compile) commit.
+
+## Build notes
+
+After #847, `kafka.server.ReplicaManager` on 4.1 is too big for SpotBugs, and `:core:spotbugsMain` fails with `SKIPPED_CLASS_TOO_BIG`. The class is about 400 lines longer on 4.1 than on 4.2 and 4.3. A trial upgrade to SpotBugs 4.9.8 with plugin 6.2.3 fails the same way. The sync(compile) commit adds an exclude filter entry for that pattern on `ReplicaManager` only. `:core:spotbugsMain` passes with it.
+
+## CI notes
+
+The first CI run after the push failed in `:storage:inkless:integrationTest`. Testcontainers couldn't find a Docker environment on the `ubuntu-26.04` runner image. 4.1 declared `testcontainers` twice, and the stale 1.20.3 entry overrode the 1.21.4 entry from #509. A follow-up commit on 4.1 removes the 1.20.3 entry. The cherry-picked commits didn't cause the failure.