Repository navigation
fix(offsets): pace the hard-cap log lines -- one WARN per pacing period, silent pauses inside a 60 s re-arm window, one summary per minute (Spec 01.05 section 3.4 item 6) - #1504
Merged
Conversation
…cillates one unit at a time, so one WARN/INFO per wait was one per batch (~300 lines a minute, 990 WARNs into the error log in seven minutes on the first deployment that met the cap); now one WARN per pacing period, pauses inside a 60 s re-arm window counted not logged, one summary INFO per minute while paced, one 'pacing ended' line after a quiet gap or on reset() (Spec 01.05 section 3.4 item 6)
…lands 52 s into the period -- at 61 s the per-minute summary fires by design, which the test had mistaken for a stray line
This was referenced Sep 26, 2026
minguyen9988
added a commit
that referenced
this pull request
Sep 26, 2026
… one squashed commit with the same content as the two pacing commits this branch was stacked on; the three conflicts (DebeziumOffsetManagement.java, spec 01.05, spec 09.01) resolve to this branch's versions, which carry the pacing text plus the byte-bound additions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defect
The hard cap on rows handed off and not yet acknowledged (#1503) holds the reader until the writers acknowledge the head of the FIFO, and one acknowledgement is exactly one unit: the reader is released, hands off the next unit, meets the cap again. At the cap it oscillates one batch at a time by construction, so the rule 'one WARN when a wait begins, one INFO when it ends' became one WARN/INFO pair per batch. Measured on the first deployment that met the cap: about 300 Handoff hard cap lines a minute and 990 WARNs into the error log in seven minutes, while the cap was doing exactly its job and nothing was wrong.
Rule (spec 01.05 section 3.4 item 6)
A pacing period. ONE WARN when a wait begins more than 60 s after the previous release (or with no previous release), naming rows, units and cap, and ONE INFO when that first wait ends. Every later wait that begins within 60 s of the previous release continues the period and is counted, not logged. ONE INFO summary per 60 s while the period lasts: pauses and milliseconds paused since the previous line and since the period began, rows and units outstanding. ONE INFO 'pacing ended' when the next wait begins after a quiet gap longer than 60 s, or on reset(). Nothing per slice, nothing when the cap is not met. The waits themselves, the dead-worker check between slices and the wait limit are unchanged: pacing changes what is logged, never how long the reader waits.
Change
Checks on this branch