Repository navigation
fix(inkless:retention): require remote copy on consolidating topics [KC-552] - #809
Merged
Merged
Conversation
viktorsomogyi
force-pushed
the
svv/KC-552-retention-guard
branch
from
September 23, 2026 09:44
65114ee to
e489ab5
Compare
viktorsomogyi
force-pushed
the
svv/KC-552-remote-log-copy-disable
branch
from
September 23, 2026 09:45
d7ba5fa to
b9b0e39
Compare
… [KC-552] `ConsolidatedDisklessLogPruner` deletes WAL batches only after remote storage confirms them. `remote.log.copy.disable=true` means that confirmation never arrives, so the pruner deletes nothing. The parent commit skips `RetentionEnforcer` for every consolidating topic, and a copy-disabled consolidating topic then has no deleter. `LogConfig` rejects an update that creates this state: remote copy turned off on a topic that will consolidate, or consolidation turned on while copy is already off. A topic that already stores `remote.log.copy.disable=true` together with consolidation can change other configs, delete that key, or set it to `false`. `createTopic` writes `diskless.enable` from the broker default `log.diskless.enable` after `LogConfig` runs. A create that sets only `remote.log.copy.disable=true` would be stored as a consolidating topic. `createTopic` rejects that create when consolidation is enabled and the topic isn't a system topic. `RetentionEnforcer` deletes WAL batches for a consolidating topic whose remote copy is disabled. ## Test plan - `LogConfigTest` covers deleting `remote.log.copy.disable`, a `retention.ms` alter that keeps the stored `true`, a broker `log.diskless.enable` default, and a classic-to-diskless switch that drops the key. - `ReplicationControlManagerInklessTest` covers a create that uses `log.diskless.enable`, an explicit `diskless.enable=true`, consolidation disabled, and a system topic. - `RetentionEnforcerTest` checks that a consolidating partition with remote copy disabled is sent for WAL retention. Assisted-by: Grok 4.7 Co-authored-by: Cursor <cursoragent@cursor.com>
viktorsomogyi
force-pushed
the
svv/KC-552-remote-log-copy-disable
branch
from
September 23, 2026 12:48
b9b0e39 to
0362e4a
Compare
viktorsomogyi
marked this pull request as ready for review
September 23, 2026 12:48
EelisK
approved these changes
Sep 23, 2026
giuseppelillo
pushed a commit
that referenced
this pull request
Sep 24, 2026
… [KC-552] (#809) `ConsolidatedDisklessLogPruner` deletes WAL batches only after remote storage confirms them. `remote.log.copy.disable=true` means that confirmation never arrives, so the pruner deletes nothing. The parent commit skips `RetentionEnforcer` for every consolidating topic, and a copy-disabled consolidating topic then has no deleter. `LogConfig` rejects an update that creates this state: remote copy turned off on a topic that will consolidate, or consolidation turned on while copy is already off. A topic that already stores `remote.log.copy.disable=true` together with consolidation can change other configs, delete that key, or set it to `false`. `createTopic` writes `diskless.enable` from the broker default `log.diskless.enable` after `LogConfig` runs. A create that sets only `remote.log.copy.disable=true` would be stored as a consolidating topic. `createTopic` rejects that create when consolidation is enabled and the topic isn't a system topic. `RetentionEnforcer` deletes WAL batches for a consolidating topic whose remote copy is disabled. ## Test plan - `LogConfigTest` covers deleting `remote.log.copy.disable`, a `retention.ms` alter that keeps the stored `true`, a broker `log.diskless.enable` default, and a classic-to-diskless switch that drops the key. - `ReplicationControlManagerInklessTest` covers a create that uses `log.diskless.enable`, an explicit `diskless.enable=true`, consolidation disabled, and a system topic. - `RetentionEnforcerTest` checks that a consolidating partition with remote copy disabled is sent for WAL retention. Assisted-by: Grok 4.7 Co-authored-by: Cursor <cursoragent@cursor.com>
giuseppelillo
pushed a commit
that referenced
this pull request
Sep 24, 2026
… [KC-552] (#809) `ConsolidatedDisklessLogPruner` deletes WAL batches only after remote storage confirms them. `remote.log.copy.disable=true` means that confirmation never arrives, so the pruner deletes nothing. The parent commit skips `RetentionEnforcer` for every consolidating topic, and a copy-disabled consolidating topic then has no deleter. `LogConfig` rejects an update that creates this state: remote copy turned off on a topic that will consolidate, or consolidation turned on while copy is already off. A topic that already stores `remote.log.copy.disable=true` together with consolidation can change other configs, delete that key, or set it to `false`. `createTopic` writes `diskless.enable` from the broker default `log.diskless.enable` after `LogConfig` runs. A create that sets only `remote.log.copy.disable=true` would be stored as a consolidating topic. `createTopic` rejects that create when consolidation is enabled and the topic isn't a system topic. `RetentionEnforcer` deletes WAL batches for a consolidating topic whose remote copy is disabled. ## Test plan - `LogConfigTest` covers deleting `remote.log.copy.disable`, a `retention.ms` alter that keeps the stored `true`, a broker `log.diskless.enable` default, and a classic-to-diskless switch that drops the key. - `ReplicationControlManagerInklessTest` covers a create that uses `log.diskless.enable`, an explicit `diskless.enable=true`, consolidation disabled, and a system topic. - `RetentionEnforcerTest` checks that a consolidating partition with remote copy disabled is sent for WAL retention. Assisted-by: Grok 4.7 Co-authored-by: Cursor <cursoragent@cursor.com>
giuseppelillo
pushed a commit
that referenced
this pull request
Sep 24, 2026
… [KC-552] (#809) `ConsolidatedDisklessLogPruner` deletes WAL batches only after remote storage confirms them. `remote.log.copy.disable=true` means that confirmation never arrives, so the pruner deletes nothing. The parent commit skips `RetentionEnforcer` for every consolidating topic, and a copy-disabled consolidating topic then has no deleter. `LogConfig` rejects an update that creates this state: remote copy turned off on a topic that will consolidate, or consolidation turned on while copy is already off. A topic that already stores `remote.log.copy.disable=true` together with consolidation can change other configs, delete that key, or set it to `false`. `createTopic` writes `diskless.enable` from the broker default `log.diskless.enable` after `LogConfig` runs. A create that sets only `remote.log.copy.disable=true` would be stored as a consolidating topic. `createTopic` rejects that create when consolidation is enabled and the topic isn't a system topic. `RetentionEnforcer` deletes WAL batches for a consolidating topic whose remote copy is disabled. ## Test plan - `LogConfigTest` covers deleting `remote.log.copy.disable`, a `retention.ms` alter that keeps the stored `true`, a broker `log.diskless.enable` default, and a classic-to-diskless switch that drops the key. - `ReplicationControlManagerInklessTest` covers a create that uses `log.diskless.enable`, an explicit `diskless.enable=true`, consolidation disabled, and a system topic. - `RetentionEnforcerTest` checks that a consolidating partition with remote copy disabled is sent for WAL retention. Assisted-by: Grok 4.7 Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ConsolidatedDisklessLogPrunerdeletes WAL batches only after remote storage confirms them.remote.log.copy.disable=truemeans that confirmation never arrives, so the pruner deletes nothing. The parent commit skipsRetentionEnforcerfor every consolidating topic, and a copy-disabled consolidating topic then has no deleter.LogConfigrejects an update that creates this state: remote copy turned off on a topic that will consolidate, or consolidation turned on while copy is already off. A topic that already storesremote.log.copy.disable=truetogether with consolidation can change other configs, delete that key, or set it tofalse.createTopicwritesdiskless.enablefrom the broker defaultlog.diskless.enableafterLogConfigruns. A create that sets onlyremote.log.copy.disable=truewould be stored as a consolidating topic.createTopicrejects that create when consolidation is enabled and the topic isn't a system topic.RetentionEnforcerdeletes WAL batches for a consolidating topic whose remote copy is disabled.Test plan
LogConfigTestcovers deletingremote.log.copy.disable, aretention.msalter that keeps the storedtrue, a brokerlog.diskless.enabledefault, and a classic-to-diskless switch that drops the key.ReplicationControlManagerInklessTestcovers a create that useslog.diskless.enable, an explicitdiskless.enable=true, consolidation disabled, and a system topic.RetentionEnforcerTestchecks that a consolidating partition with remote copy disabled is sent for WAL retention.Assisted-by: Grok 4.7
Co-authored-by: Cursor cursoragent@cursor.com