Hi Mo, Is the broker id in your server properties really the same for all 3, or is that a typo?
Steve On Wed, Mar 19, 2014 at 8:10 AM, Jun Rao <jun...@gmail.com> wrote: > Were there any logging before that? There should be some logging that tells > you why the rebalance failed. > > Thanks, > > Jun > > > On Wed, Mar 19, 2014 at 7:07 AM, Mo Firouz <mo.fir...@visualdna.com> > wrote: > > > Hello. > > > > I am trying to migrate from Kafka 0.7 to 0.8, and the first stage of this > > is to use the Migration Tool. > > > > I have setup a local Kafka 0.8 cluster consisting of 3 nodes on my local > > Vagrant (along with a zookeeper running locally). The Migration Tool > > Producer will connect to these three nodes in order to replicate the > data. > > The Migration Tool Consumer(s) will connect to my Staging boxes to > consume > > Kafka 0.7 events. > > > > However, upon running the Migration Tool, I'll get the following error: > > > > *vagrant@precise64:~/Desktop$ > > > ~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh > > > kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar > > > --zkclient.01.jar zkclient-0.2.0.jar --num.producers 16 > > > --producer.config=/home/vagrant/Deskt/producerConfig.properties > > > --consumer.config=/home/vagrant/Desktop/consumerConfig.properties > > > --whitelist "prod_conversions"* > > > > [2014-03-19 13:14:38,126] WARN Property mirror.topics.whitelist is not > > > valid (kafka.utils.VerifiableProperties) > > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > > > SLF4J: Defaulting to no-operation (NOP) logger implementation > > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for > > further > > > details. > > > Kafka migration tool failed due to: > > > java.lang.reflect.InvocationTargetException > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) > > > Caused by: kafka.common.ConsumerRebalanceFailedException: > > > mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't > > > rebalance after 4 retries > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) > > > at > > > > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) > > > at > > > > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) > > > ... 5 more > > > [2014-03-19 13:14:47,082] ERROR Kafka migration tool failed: > > > (kafka.tools.KafkaMigrationTool) > > > java.lang.reflect.InvocationTargetException > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) > > > Caused by: kafka.common.ConsumerRebalanceFailedException: > > > mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't > > > rebalance after 4 retries > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) > > > at > > > > > > kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) > > > at > > > > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) > > > at > > > > > > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) > > > ... 5 more > > > > > > My 3 *Kafka server* configurations are as follows: > > > > *cat kafka-server1.properties * > > > zookeeper.connect=localhost:2181 > > > zookeeper.connection.timeout.ms=1000000 > > > broker.id=1 > > > port=9092 > > > log.dir=/tmp/kafka-logs-1 > > > > > > > > > > > > *cat kafka-server2.properties * > > > zookeeper.connect=localhost:2181 > > > zookeeper.connection.timeout.ms=1000000 > > > broker.id=1 > > > port=9093 > > > log.dir=/tmp/kafka-logs-2 > > > > > > > > > *cat kafka-server3.properties * > > > zookeeper.connect=localhost:2181 > > > zookeeper.connection.timeout.ms=1000000 > > > broker.id=1 > > > port=9094 > > > log.dir=/tmp/kafka-logs-3 > > > > > > My *consumer* configuration is: > > > > *cat consumerConfig.properties * > > > group.id = mfirouz-vagrant-kafkamigration > > > zookeeper.connect = zk-01.s.visualdna.com:2181, > > zk-02.s.visualdna.com:2181, > > > zk-03.s.visualdna.com:2181 > > > mirror.topics.whitelist = "prod_conversions" > > > > > > And finally, my *producer* configuration is: > > > > *cat producerConfig.properties* > > > zk.connect = localhost:2181 > > > whitelist = "prod_conversions" > > > producer.type = "sync" > > > request.required.acks = 1 > > > metadata.broker.list = localhost:9092,localhost:9093,localhost:9094 > > > > > > I was wondering if anyone had any insight about this and how to fix it? > > > > Thanks a lot in advance > > Mo. > > > -- Steve Robenalt Software Architect HighWire | Stanford University 425 Broadway St, Redwood City, CA 94063 srobe...@stanford.edu http://highwire.stanford.edu
