Were there any logging before that? There should be some logging that tells you why the rebalance failed.
Thanks, Jun On Wed, Mar 19, 2014 at 7:07 AM, Mo Firouz <mo.fir...@visualdna.com> wrote: > Hello. > > I am trying to migrate from Kafka 0.7 to 0.8, and the first stage of this > is to use the Migration Tool. > > I have setup a local Kafka 0.8 cluster consisting of 3 nodes on my local > Vagrant (along with a zookeeper running locally). The Migration Tool > Producer will connect to these three nodes in order to replicate the data. > The Migration Tool Consumer(s) will connect to my Staging boxes to consume > Kafka 0.7 events. > > However, upon running the Migration Tool, I'll get the following error: > > *vagrant@precise64:~/Desktop$ > > ~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh > > kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar > > --zkclient.01.jar zkclient-0.2.0.jar --num.producers 16 > > --producer.config=/home/vagrant/Deskt/producerConfig.properties > > --consumer.config=/home/vagrant/Desktop/consumerConfig.properties > > --whitelist "prod_conversions"* > > [2014-03-19 13:14:38,126] WARN Property mirror.topics.whitelist is not > > valid (kafka.utils.VerifiableProperties) > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > > SLF4J: Defaulting to no-operation (NOP) logger implementation > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for > further > > details. > > Kafka migration tool failed due to: > > java.lang.reflect.InvocationTargetException > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:606) > > at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) > > Caused by: kafka.common.ConsumerRebalanceFailedException: > > mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't > > rebalance after 4 retries > > at > > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) > > at > > > kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) > > at > > > kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) > > at > > > kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) > > at > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) > > at > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) > > ... 5 more > > [2014-03-19 13:14:47,082] ERROR Kafka migration tool failed: > > (kafka.tools.KafkaMigrationTool) > > java.lang.reflect.InvocationTargetException > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:606) > > at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) > > Caused by: kafka.common.ConsumerRebalanceFailedException: > > mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't > > rebalance after 4 retries > > at > > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) > > at > > > kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) > > at > > > kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) > > at > > > kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) > > at > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) > > at > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) > > ... 5 more > > > My 3 *Kafka server* configurations are as follows: > > *cat kafka-server1.properties * > > zookeeper.connect=localhost:2181 > > zookeeper.connection.timeout.ms=1000000 > > broker.id=1 > > port=9092 > > log.dir=/tmp/kafka-logs-1 > > > > > > > *cat kafka-server2.properties * > > zookeeper.connect=localhost:2181 > > zookeeper.connection.timeout.ms=1000000 > > broker.id=1 > > port=9093 > > log.dir=/tmp/kafka-logs-2 > > > > > *cat kafka-server3.properties * > > zookeeper.connect=localhost:2181 > > zookeeper.connection.timeout.ms=1000000 > > broker.id=1 > > port=9094 > > log.dir=/tmp/kafka-logs-3 > > > My *consumer* configuration is: > > *cat consumerConfig.properties * > > group.id = mfirouz-vagrant-kafkamigration > > zookeeper.connect = zk-01.s.visualdna.com:2181, > zk-02.s.visualdna.com:2181, > > zk-03.s.visualdna.com:2181 > > mirror.topics.whitelist = "prod_conversions" > > > And finally, my *producer* configuration is: > > *cat producerConfig.properties* > > zk.connect = localhost:2181 > > whitelist = "prod_conversions" > > producer.type = "sync" > > request.required.acks = 1 > > metadata.broker.list = localhost:9092,localhost:9093,localhost:9094 > > > I was wondering if anyone had any insight about this and how to fix it? > > Thanks a lot in advance > Mo. >
