Hi Jun, Thanks for your response and help. Yes indeed BOTH the kafka and zookeeper variations were different that was causing this error!
It's now working! Thanks for you help and time! Cheers, Mo. On 19 March 2014 14:07, Mo Firouz <mo.fir...@visualdna.com> wrote: > Hello. > > I am trying to migrate from Kafka 0.7 to 0.8, and the first stage of this > is to use the Migration Tool. > > I have setup a local Kafka 0.8 cluster consisting of 3 nodes on my local > Vagrant (along with a zookeeper running locally). The Migration Tool > Producer will connect to these three nodes in order to replicate the data. > The Migration Tool Consumer(s) will connect to my Staging boxes to consume > Kafka 0.7 events. > > However, upon running the Migration Tool, I'll get the following error: > > *vagrant@precise64:~/Desktop$ >> ~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh >> kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar >> --zkclient.01.jar zkclient-0.2.0.jar --num.producers 16 >> --producer.config=/home/vagrant/Deskt/producerConfig.properties >> --consumer.config=/home/vagrant/Desktop/consumerConfig.properties >> --whitelist "prod_conversions"* > > [2014-03-19 13:14:38,126] WARN Property mirror.topics.whitelist is not >> valid (kafka.utils.VerifiableProperties) >> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". >> SLF4J: Defaulting to no-operation (NOP) logger implementation >> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for >> further details. >> Kafka migration tool failed due to: >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) >> Caused by: kafka.common.ConsumerRebalanceFailedException: >> mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't >> rebalance after 4 retries >> at >> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) >> at >> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) >> at >> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) >> at >> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) >> ... 5 more >> [2014-03-19 13:14:47,082] ERROR Kafka migration tool failed: >> (kafka.tools.KafkaMigrationTool) >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) >> Caused by: kafka.common.ConsumerRebalanceFailedException: >> mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't >> rebalance after 4 retries >> at >> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) >> at >> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) >> at >> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) >> at >> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) >> ... 5 more > > > My 3 *Kafka server* configurations are as follows: > > *cat kafka-server1.properties * >> zookeeper.connect=localhost:2181 >> zookeeper.connection.timeout.ms=1000000 >> broker.id=1 >> port=9092 >> log.dir=/tmp/kafka-logs-1 > > > >> >> * cat kafka-server2.properties * >> zookeeper.connect=localhost:2181 >> zookeeper.connection.timeout.ms=1000000 >> broker.id=1 >> port=9093 >> log.dir=/tmp/kafka-logs-2 > > > >> *cat kafka-server3.properties * >> zookeeper.connect=localhost:2181 >> zookeeper.connection.timeout.ms=1000000 >> broker.id=1 >> port=9094 >> log.dir=/tmp/kafka-logs-3 > > > My *consumer* configuration is: > > *cat consumerConfig.properties * >> group.id = mfirouz-vagrant-kafkamigration >> zookeeper.connect = zk-01.s.visualdna.com:2181,zk-02.s.visualdna.com:2181 >> ,zk-03.s.visualdna.com:2181 >> mirror.topics.whitelist = "prod_conversions" > > > And finally, my *producer* configuration is: > > *cat producerConfig.properties* >> zk.connect = localhost:2181 >> whitelist = "prod_conversions" >> producer.type = "sync" >> request.required.acks = 1 >> metadata.broker.list = localhost:9092,localhost:9093,localhost:9094 > > > I was wondering if anyone had any insight about this and how to fix it? > > Thanks a lot in advance > Mo. >
