Re: Kafka High Level Consumer Fail Over

2014-06-13 Thread Guozhang Wang
I did not see any attachments in either emails? Guozhang On Fri, Jun 13, 2014 at 1:36 PM, Bhavesh Mistry wrote: > HI Guozhang, > > We have own monitoring tool which get the data from the Zookeeper and you > can see the attached screen which list no owner, but back consumer are not > picking up

Re: Kafka High Level Consumer Fail Over

2014-06-13 Thread Bhavesh Mistry
HI Guozhang, We have own monitoring tool which get the data from the Zookeeper and you can see the attached screen which list no owner, but back consumer are not picking up the partition ? I had restart the java process on machine which had the above issue. Also, my previous email I have send y

Re: Kafka High Level Consumer Fail Over

2014-06-13 Thread Guozhang Wang
They should automatically pick up the partition with no owner. Could you use kafka.tool.ConsumerOffsetChecker to verify which partition does not have an owner, and check the logs of the back-up consumers for rebalance process, any exceptions/warning there? Guozhang On Fri, Jun 13, 2014 at 12:34

Re: Kafka High Level Consumer Fail Over

2014-06-13 Thread Bhavesh Mistry
We have 3 node cluster separate physical box for consumer group and consumer that died "mupd_logmon_hb_ events_sdc-q1-logstream-8-1402448850475-6521f70a". On the box, I show the above Exception. What can I configure such way, that when a partition in COnsumer Group does not have "Owner" other

Kafka High Level Consumer Fail Over

2014-06-12 Thread Bhavesh Mistry
Hi Kafka Dev Team/ Users, We have high level consumer group consuming from 32 partitions for a topic. We have been running 48 consumers in this group across multiple servers. We have kept 16 as back-up consumers, and hoping when the consumer dies, meaning when Zookeeper does not have an owner