Hi Richard, Thanks for the KIP. I am a little confused on what is proposed. The KIP suggests that after recovery from a consumer crash, there will be two consumers consuming from the same partition. One consumes starting from the log end offset at the point of recovery, and another consumes starting from the last committed offset and keeping consuming with the first consumer in parallel? Does that mean the messages after the recovery point will be consumed twice? If those two consumer commits offsets, does that mean the committed offsets may rewind?
The proposal sounds a little hacky and introduce some non-deterministic behavior. It would be useful to have a concrete use case example to explain what is actually needed. If the goal is to reduce the number of records that are reprocessed when consume crashes, maybe we can have an auto commit interval based on number of messages. If the application just wants to read from the end of the log after recovery from crash, would calling seekToEnd explicitly work? Thanks, Jiangjie (Becket) Qin On Thu, Jul 5, 2018 at 6:46 PM, Richard Yu <yohan.richard...@gmail.com> wrote: > Hi all, > > I would like to discuss KIP-333 (which proposes a faster mode of > rebalancing). > Here is the link for the KIP: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 333%3A+Add+faster+mode+of+rebalancing > > Thanks, > Richard Yu >
