high level consumer shutdown exception

2014-04-19 Thread Yonghui Zhao
In kafka 0.8.1, when I shut down high level consumer, I found one exception. I think it is expected, right? 14/04/19 21:12:47 INFO consumer.ZookeeperConsumerConnector: [1_yozhao-ubuntu-1397913156352-9b13f962], ZKConsumerConnector shutting down 14/04/19 21:12:47 INFO zookeeper.ClientCnxn: Client s

Re: too many rebalances in my consumer log and fetcher threads are getting stopped

2014-04-19 Thread ankit tyagi
Hi Jun, i grabbed for 'begin rebalancing consumer' in my log and got around *5000*times in today's logs. Can there be any other reason instead full GC to initiate re-balances. Are there any broker properties which can be tune to reduce rebalances? On Sat, Apr 19, 2014 at 9:31 AM, Jun Rao wrot

Re: Cluster design distribution and JBOD vs RAID

2014-04-19 Thread Jay Kreps
I think we are saying the same thing. If any drive fails the broker is down. But when the drive is repaired only the data on the destroyed drive will need to be restored from replicas. -Jay On Fri, Apr 18, 2014 at 3:24 PM, Maxime Brugidou wrote: > Are you sure about that? Our latest tests show