Re: Kafka Replication Behavior

2014-04-28 Thread Hanish Bansal
Thanks Bert for response. Are you saying that if i set replication factor to 3 then two nodes can go down, in that case only 3rd down can serve the data? In that case(replication factor 3 ) also if some partitions are in synchronization process and meanwhile a node goes down then we may face this

Re: Kafka Replication Behavior

2014-04-28 Thread Bert Corderman
Only a single broker needs to be online for data to be available. In your example partition 2 and 3 had copies of data on broker 0 and 1. When those two brokers went down your data was unavailable. To withstand two brokers going offline you would want to change your replication factor to 3. O

Re: Kafka Replication Behavior

2014-04-28 Thread Hanish Bansal
After restarting it start working fine. What i understand from this scenario is: There must be atleast 2 brokers in isr (In Sync Replica) list of each topic partition before any node goes down which has replica, otherwise topic may go to unstable state. Or In other words, If topic partitions are

Re: Kafka Replication Behavior

2014-04-28 Thread Hanish Bansal
After restarting it start working fine. What i understand from this scenario is: On Mon, Apr 28, 2014 at 8:30 PM, Jun Rao wrote: > Partition 2 and 3 are not accessible since both replicas are down. Have you > restarted the 2 failed brokers? > > Thanks, > > Jun > > > On Sun, Apr 27, 2014 at 11

Re: Kafka Replication Behavior

2014-04-28 Thread Jun Rao
Partition 2 and 3 are not accessible since both replicas are down. Have you restarted the 2 failed brokers? Thanks, Jun On Sun, Apr 27, 2014 at 11:58 PM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > Hi All, > > I have 3 nodes kafka cluster with below configuration: > > Replication

Kafka Replication Behavior

2014-04-27 Thread Hanish Bansal
Hi All, I have 3 nodes kafka cluster with below configuration: Replication factor: 2 Number of partitions: 6 topic-name: notification As soon as data is ingested into notification topic, 2 nodes goes down. After that i am not able to push the data into that topic. When i check the status using