Hello. I am trying to migrate from Kafka 0.7 to 0.8, and the first stage of this is to use the Migration Tool.
I have setup a local Kafka 0.8 cluster consisting of 3 nodes on my local Vagrant (along with a zookeeper running locally). The Migration Tool Producer will connect to these three nodes in order to replicate the data. The Migration Tool Consumer(s) will connect to my Staging boxes to consume Kafka 0.7 events. However, upon running the Migration Tool, I'll get the following error: *vagrant@precise64:~/Desktop$ > ~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh > kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar > --zkclient.01.jar zkclient-0.2.0.jar --num.producers 16 > --producer.config=/home/vagrant/Deskt/producerConfig.properties > --consumer.config=/home/vagrant/Desktop/consumerConfig.properties > --whitelist "prod_conversions"* [2014-03-19 13:14:38,126] WARN Property mirror.topics.whitelist is not > valid (kafka.utils.VerifiableProperties) > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > SLF4J: Defaulting to no-operation (NOP) logger implementation > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further > details. > Kafka migration tool failed due to: > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) > Caused by: kafka.common.ConsumerRebalanceFailedException: > mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't > rebalance after 4 retries > at > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) > at > kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) > at > kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) > at > kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) > at > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) > at > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) > ... 5 more > [2014-03-19 13:14:47,082] ERROR Kafka migration tool failed: > (kafka.tools.KafkaMigrationTool) > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) > Caused by: kafka.common.ConsumerRebalanceFailedException: > mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't > rebalance after 4 retries > at > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) > at > kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) > at > kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) > at > kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) > at > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) > at > kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) > ... 5 more My 3 *Kafka server* configurations are as follows: *cat kafka-server1.properties * > zookeeper.connect=localhost:2181 > zookeeper.connection.timeout.ms=1000000 > broker.id=1 > port=9092 > log.dir=/tmp/kafka-logs-1 > > *cat kafka-server2.properties * > zookeeper.connect=localhost:2181 > zookeeper.connection.timeout.ms=1000000 > broker.id=1 > port=9093 > log.dir=/tmp/kafka-logs-2 > *cat kafka-server3.properties * > zookeeper.connect=localhost:2181 > zookeeper.connection.timeout.ms=1000000 > broker.id=1 > port=9094 > log.dir=/tmp/kafka-logs-3 My *consumer* configuration is: *cat consumerConfig.properties * > group.id = mfirouz-vagrant-kafkamigration > zookeeper.connect = zk-01.s.visualdna.com:2181,zk-02.s.visualdna.com:2181, > zk-03.s.visualdna.com:2181 > mirror.topics.whitelist = "prod_conversions" And finally, my *producer* configuration is: *cat producerConfig.properties* > zk.connect = localhost:2181 > whitelist = "prod_conversions" > producer.type = "sync" > request.required.acks = 1 > metadata.broker.list = localhost:9092,localhost:9093,localhost:9094 I was wondering if anyone had any insight about this and how to fix it? Thanks a lot in advance Mo.
