> On Jul 21, 2015, at 9:15 AM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > > On Tue, Jul 21, 2015 at 2:38 AM, Stevo Slavić <ssla...@gmail.com> wrote: > >> Hello Apache Kafka community, >> >> I find new consumer poll/seek javadoc a bit confusing. Just by reading docs >> I'm not sure what the outcome will be, what is expected in following >> scenario: >> >> - kafkaConsumer is instantiated with auto-commit off >> - kafkaConsumer.subscribe(someTopic) >> - kafkaConsumer.position is called for every TopicPartition HLC is actively >> subscribed on >> >> and then when doing multiple poll calls in succession (without calling >> commit), does seek have to be called in between poll calls to position HLC >> to skip what was read in previous poll, or does HLC keep that state >> (position after poll) in memory, so that next poll (without seek in between >> two poll calls) will continue from where last poll stopped? >> > > The position is tracked in-memory within the consumer, so as long as there > isn't a consumer rebalance, consumption will just proceed with subsequent > messages (i.e. the behavior I think most people would find intuitive). > However, if a rebalance occurs (another consumer instance joins the group > or some leave), then a partition may be assigned to an different consumer > instance that has no idea about the current position and will restart based > on the offset reset setting (because attempting to fetch the committed > offset will fail since no offsets have been committed). >
Ewen, What happens if there is a broker failure and a new broker becomes the partition leader? Does the high level consumer start listening to the new partition leader at the in-memory position, or does it restart based on saved offsets? Thanks, -James > -Ewen > > >> Could be it's just me not understanding this from javadoc. If not, maybe >> javadoc can be improved to make this (even) more obvious. >> >> Kind regards, >> Stevo Slavic. >> > > > > -- > Thanks, > Ewen
