Repository navigation
Tags
Tags
[LI-HOTFIX] Add OneAboveMinIsrCounter sensor. (linkedin#71) TICKET = LI_DESCRIPTION = Add the OneAboveMinIsrCounter metrics to help determining the good timing of broker rolling bounce. EXIT_CRITERIA = MANUAL [""]
[LI-HOTFIX]
Added support to enable passthrough mirroring data from V1 format to V2 format (linkedin#70)
Pass through mirroring as it is today, works when the log.message.format.version between source and destination cluster is the same (0.10.0 to 0.10.0, 2.0 to 2.0). When there is a mixed message format between source and destination, KMM throws errors.
This patch makes passthrough mirroring work when the source is at 0.10.0 format and destination is at 2.0 format.
This patch does not address the errors/issues when the source is at 2.0 format and destination is at 0.10.0 format.
TICKET =
[LI-HOTFIX] Be ultra lazy in loading lazy indices during index deleti… …on. (linkedin#68) TICKET = LI_DESCRIPTION = Avoid initialization of logSegment indices during deleteIfExists in case the logSegment indices have not been initialized before. This helps with overall resource usage of Kafka server, as well as prevents flaky errors while deleting log segments due to java.io.IOException: No such file or directory. EXIT_CRITERIA = MANUAL [""]
[LI-CHERRY-PICK][dca1bf1] Fix LazyTimeIndex & LazyOffsetIndex concurr… …ency (linkedin#67) TICKET = KAFKA-9156 LI_DESCRIPTION = This patch fixes BufferOverflow exceptions due to concurrent accesses to indices. EXIT_CRITERIA = HASH [dca1bf1] ORIGINAL_DESCRIPTION = Race condition in concurrent `get` method invocation of lazy indexes might lead to multiple `OffsetIndex`/`TimeIndex` objects being concurrently created. When that happens position of `MappedByteBuffer` in `AbstractIndex` is advanced to the last entry which in turn leads to a critical `BufferOverflowException` being thrown whenever leader or replica tries to append a record to the segment. Moreover, `file_=` setter is seemingly also vulnerable to the race, since multiple threads can attempt to set a new file reference as well as create new Time/OffsetIndex objects at the same time. This might lead to the discrepant File references being held by e.g. LazyTimeIndex and its TimeIndex counterpart. This patch attempts to fix the issue by making sure that index objects are atomically constructed when loaded lazily via `get` method. Additionally, `file` reference modifications are also made atomic and thread safe. Note that the `Lazy*Index` mutation operations are executed with a lock held by the callers, but `get` can be called without a lock (e.g. from `Log.read`). Reviewers: Jun Rao <junrao@gmail.com>, Jason Gustafson <jason@confluent.io>, Shilin Lu, Ismael Juma <ismael@juma.me.uk> Co-authored-by: Alex Mironov <alexandrfox@gmail.com>
[LI-HOTFIX] Improve shutdown performance via lazy accessing the offse… …t and time indices. (linkedin#65) TICKET = LI_DESCRIPTION = KAFKA-7283 enabled lazy mmap on index files by initializing indices on-demand rather than performing costly disk/memory operations when creating all indices on broker startup. This helped reducing the startup time of brokers. However, segment indices are still created on closing segments, regardless of whether they needs to be closed or not. This patch: * Improves shutdown performance via lazy accessing the offset and time indices. * Eliminates redundant disk accesses and memory mapped operations while deleting or renaming files that back segment indices. * Prevents illegal accesses to underlying indices of a closed segment, which would lead to memory leaks due to recreation of the underlying memory mapped objects. EXIT_CRITERIA = MANUAL [""]
[LI-HOTFIX] Avoid excessive info logs in the client side for incremen… …tal fetch and print out more informative logs for throttling (linkedin#48) * [LI-HOTFIX] Avoid excessive info logs in the client side for incremental fetch and print out more informative logs for throttling TICKET = KAFKA-8921 LI_DESCRIPTION = Currently in FetchSessionHandler::handleResponse, the following info log will get printed out excessively when the session is evicted from the server-side cache even though there is nothing wrong with the fetch request and client cannot do much to improve it. ``` Node xxx was unable to process the fetch request with (sessionId=xxx, epoch=xxx): FETCH_SESSION_ID_NOT_FOUND. ``` Moreover, when the fetch request gets throttled, the following info logs will also get printed out, which are very misleading. ``` Node xxx sent an invalid full fetch response with ... Node xxx sent an invalid incremental fetch response with ... ``` We should avoid logging these things in INFO level and print out more informative logs for throttling. EXIT_CRITERIA = MANUAL ["When the hotfix is pushed to apache/kafka"]
Add counters for the following: (linkedin#59) 1. Total number of incoming messages 2. Total number of incoming bytes
PreviousNext