Re: Understanding how producers and consumers behave in case of node failures in 0.8

I am planning to use kafka 0.8 spout and after studing the source code found that it doesnt handle errors. There is a fork that adds try catch over using fetchResponse but my guess is this will lead to spout attempting the same partition infinitely until the leader is elected/comes back online. I w

Re: Understanding how producers and consumers behave in case of node failures in 0.8

Thank you for posting these guidelines. I'm wondering if anyone out there that is using the Kafka spout (for Storm) knows whether or not the Kafka spout takes care of these types of details ? regards -chris On Thu, Oct 24, 2013 at 2:05 PM, Neha Narkhede wrote: > Yes, when a leader dies, th

Re: Understanding how producers and consumers behave in case of node failures in 0.8

Yes, when a leader dies, the preference is to pick a leader from the ISR. If not, the leader is picked from any other available replica. But if no replicas are alive, the partition goes offline and all production and consumption halts, until at least one replica is brought online. Thanks, Neha O

Re: Understanding how producers and consumers behave in case of node failures in 0.8

>>publishing to and consumption from the partition will halt and will not resume until the faulty leader node recovers Can you confirm that's the case? I think they won't wait until leader recovered and will try to elect new leader from existing non-ISR replicas? And in case if they wait, and faul

Re: Understanding how producers and consumers behave in case of node failures in 0.8

Thanks Neha On 24 October 2013 18:11, Neha Narkhede wrote: > Yes. And during retries, the producer and consumer refetch metadata. > > Thanks, > Neha > On Oct 24, 2013 3:09 AM, "Aniket Bhatnagar" > wrote: > > > I am trying to understand and document how producers & consumers > > will/should beh

Re: Understanding how producers and consumers behave in case of node failures in 0.8

Yes. And during retries, the producer and consumer refetch metadata. Thanks, Neha On Oct 24, 2013 3:09 AM, "Aniket Bhatnagar" wrote: > I am trying to understand and document how producers & consumers > will/should behave in case of node failures in 0.8. I know there are > various other threads t

Understanding how producers and consumers behave in case of node failures in 0.8

I am trying to understand and document how producers & consumers will/should behave in case of node failures in 0.8. I know there are various other threads that discuss this but I wanted to bring all the information together in one post. This should help people building producers & consumers in oth