Perfect. By the way, I see you went to GT...you wouldn't happen to know a certain Lex Spoon, would you?
thanks, rob ________________________________________ From: Neha Narkhede [neha.narkh...@gmail.com] Sent: Friday, May 17, 2013 7:40 AM To: users@kafka.apache.org Subject: RE: possible to shutdown a consumerConnector without flushing the offset If you turn off auto.commit.enable, that will ensure that messages are replayed whenever a consumer starts up and rebalances. Thanks, Neha On May 17, 2013 6:35 AM, "Withers, Robert" <robert.with...@dish.com> wrote: > Certainly I will try. Our understanding is that there are 2 scenarios > where messages could be replayed: > > 1. if a consumer falls over hard, there are some message consumptions > whose offsets had not yet been flushed to zookeeper and so when a rebalance > occurs the consumer that starts getting messages from a partition that > flipped from the broke consumer will replay some messages. > 2. I think a combination of a leader election and a broker failure may > replay messages. > > We want to demonstrate the first, but our stats and correlation harness > needs to keep running. Love to demonstrate the second but egads, it's > tricky. > > thanks, > rob > ________________________________________ > From: Neha Narkhede [neha.narkh...@gmail.com] > Sent: Friday, May 17, 2013 7:29 AM > To: users@kafka.apache.org > Subject: Re: possible to shutdown a consumerConnector without flushing the > offset > > Can you provide more details about what you mean by measuring replay when > you kill a consumer? > > Thanks, > Neha > On May 17, 2013 6:26 AM, "Withers, Robert" <robert.with...@dish.com> > wrote: > > > Would it be possible for someone to provide me with a 0.8 jar that > > implements a ConsumerConnector.hardShutdown, which would interrupt all > > threads yet not do a final offset flush. We want to measure replay so we > > want to simulate a kill -9, but we want to keep running the process to > > flush stats and have them available locally as well. > > > > thanks, > > rob >
