Repository navigation
feat(inkless:metadata): add classicPrefixInRemoteStorage flag [KC-580] - #824
Merged
Merged
Conversation
The two-phase classic-to-diskless switch (KC-576) tells phase 1 from phase 2 with one per-partition flag in KRaft metadata. This change adds the field and its serialization. Nothing reads or writes the flag yet, so behavior does not change. The flag uses tag 103 on PartitionRecord and PartitionChangeRecord, next to the switch tags 100 to 102. Writers emit it only when it is true, so a cluster that never sets it writes the same records as before. An absent tag means false for a new registration and "no change" for a change record. The decoder reads only the first payload byte: 1 is true and any other value is false. It does not check the payload length and it does not throw, so a bad payload cannot fail metadata replay or trigger the deletion of the local classic prefix. A new meaning needs a new field, not a new value of this one. A change record without tag 103 keeps the flag, including a record that carries a negative seal. The controller already rejects those records once a seal is committed. KC-586 extends that guard to every seal request after the flag is set. Assisted-by: Claude Sonnet 5.5
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The flag is consistently propagated and its compatibility and serialization behavior are thoroughly tested.
0 open findings
What changed in this PR
Adds serialized metadata for tracking whether a partition’s classic prefix is in remote storage, without changing runtime behavior.
Changes:
- Adds tagged-field encoding and defensive decoding for tag
103. - Propagates the flag through
PartitionRegistration. - Adds merge, snapshot, serialization, and representation tests.
| File | Description |
|---|---|
metadata/src/main/java/org/apache/kafka/metadata/InitDisklessLogFields.java |
Defines tag 103 and its codec. |
metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java |
Stores, merges, serializes, and compares the flag. |
metadata/src/test/java/org/apache/kafka/metadata/InitDisklessLogFieldsTest.java |
Tests tag encoding and decoding. |
metadata/src/test/java/org/apache/kafka/metadata/PartitionRegistrationTest.java |
Tests registration behavior and persistence. |
metadata/src/test/java/org/apache/kafka/image/TopicsImageTest.java |
Tests snapshot and wire-format round trips. |
metadata/src/test/java/org/apache/kafka/image/node/TopicImageNodeTest.java |
Updates the expected string representation. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
jeqo
marked this pull request as ready for review
October 13, 2026 09:56
viktorsomogyi
approved these changes
Oct 13, 2026
jeqo
added a commit
that referenced
this pull request
Oct 19, 2026
#824) The two-phase classic-to-diskless switch (KC-576) tells phase 1 from phase 2 with one per-partition flag in KRaft metadata. This change adds the field and its serialization. Nothing reads or writes the flag yet, so behavior does not change. The flag uses tag 103 on PartitionRecord and PartitionChangeRecord, next to the switch tags 100 to 102. Writers emit it only when it is true, so a cluster that never sets it writes the same records as before. An absent tag means false for a new registration and "no change" for a change record. The decoder reads only the first payload byte: 1 is true and any other value is false. It does not check the payload length and it does not throw, so a bad payload cannot fail metadata replay or trigger the deletion of the local classic prefix. A new meaning needs a new field, not a new value of this one. A change record without tag 103 keeps the flag, including a record that carries a negative seal. The controller already rejects those records once a seal is committed. KC-586 extends that guard to every seal request after the flag is set. Assisted-by: Claude Sonnet 5.5
jeqo
added a commit
that referenced
this pull request
Oct 19, 2026
#824) The two-phase classic-to-diskless switch (KC-576) tells phase 1 from phase 2 with one per-partition flag in KRaft metadata. This change adds the field and its serialization. Nothing reads or writes the flag yet, so behavior does not change. The flag uses tag 103 on PartitionRecord and PartitionChangeRecord, next to the switch tags 100 to 102. Writers emit it only when it is true, so a cluster that never sets it writes the same records as before. An absent tag means false for a new registration and "no change" for a change record. The decoder reads only the first payload byte: 1 is true and any other value is false. It does not check the payload length and it does not throw, so a bad payload cannot fail metadata replay or trigger the deletion of the local classic prefix. A new meaning needs a new field, not a new value of this one. A change record without tag 103 keeps the flag, including a record that carries a negative seal. The controller already rejects those records once a seal is committed. KC-586 extends that guard to every seal request after the flag is set. Assisted-by: Claude Sonnet 5.5
jeqo
added a commit
that referenced
this pull request
Oct 19, 2026
#824) The two-phase classic-to-diskless switch (KC-576) tells phase 1 from phase 2 with one per-partition flag in KRaft metadata. This change adds the field and its serialization. Nothing reads or writes the flag yet, so behavior does not change. The flag uses tag 103 on PartitionRecord and PartitionChangeRecord, next to the switch tags 100 to 102. Writers emit it only when it is true, so a cluster that never sets it writes the same records as before. An absent tag means false for a new registration and "no change" for a change record. The decoder reads only the first payload byte: 1 is true and any other value is false. It does not check the payload length and it does not throw, so a bad payload cannot fail metadata replay or trigger the deletion of the local classic prefix. A new meaning needs a new field, not a new value of this one. A change record without tag 103 keeps the flag, including a record that carries a negative seal. The controller already rejects those records once a seal is committed. KC-586 extends that guard to every seal request after the flag is set. Assisted-by: Claude Sonnet 5.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The two-phase classic-to-diskless switch (KC-576) tells phase 1 from phase 2 with one per-partition flag in KRaft metadata. This PR adds the field and its serialization. Nothing reads or writes the flag yet, so behavior does not change.
Changes
103onPartitionRecordandPartitionChangeRecord, next to the switch tags 100 to 102. Writers emit it only when the flag istrue.PartitionRegistration.classicPrefixInRemoteStorage, defaultfalse. It is part of the builder,toRecord,merge,diff, equality, andtoString.mergewhen a change record has no tag 103, including a record with a negative seal. The controller rejects such a record once a seal is committed, and KC-586 extends that guard to every seal request after the flag is set.1is true and any other value is false. The decoder does not throw, so a bad payload cannot fail metadata replay or trigger the deletion of the local classic prefix. A new meaning needs a new field.Test plan
InitDisklessLogFieldsTestpins the encoder and the decoder: the tag number, absent versus present, and which payload bytes read as true or false.PartitionRegistrationTestcovers the snapshot path with and without the other switch tags, and checks thatmergekeeps the flag for ISR, leader, seal, and negative-seal changes.TopicsImageTestround-trips a partition with the flag through image records and through the serialized record format.Rollout and follow-ups
DescribeTopicPartitionsresponse and shows the phase in the switch tool.