Re: KafkaException: Should not set log end offset on partition

2014-12-03 Thread svante karlsson
I found some logs like this before everything started to go wrong ... [2014-12-02 07:08:11,722] WARN Partition [test3,13] on broker 2: No checkpointed highwatermark is found for partition [test3,7] (kafka.cluster.Partition) [2014-12-02 07:08:11,722] WARN Partition [test3,7] on broker 2: No checkpo

KafkaException: Should not set log end offset on partition

2014-12-03 Thread svante karlsson
I've installed (for ansible scripting testing purposes) 3 VM's each containing kafka & zookeeer clustered together Ubuntu 14.04 Zookeepers are 3.4.6 and kafka 2.11-0.8.2-beta The kafka servers have broker id's 2, 4, 6 The zookeepers seems happy. The kafka servers start up and seems happy. I can

Re: Zookeeper load burst during replication

2014-12-03 Thread Yury Ruchin
Hi Guozhang, No consumer clients were consuming when the broker was replicating, at least none that would commit offsets to ZK. As for the controller log, that's a good idea, I will try it out. Thanks! 2014-12-02 22:28 GMT+03:00 Guozhang Wang : > Kafka brokers uses ZK for metadata storage, and K