Hi Jun, Thanks for the reply.
Please find below the DEBUG output of the log. *vagrant@precise64:~/Desktop$ ~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar --zkclient.01.jar zkclient-0.2.0.jar --num.producers 16 --producer.config=/home/vagrant/Desktop/producerConfig.properties --consumer.config=/home/vagrant/Desktop/consumerConfig.properties --whitelist "prod_conversions"* [2014-03-19 17:35:29,856] INFO Verifying properties (kafka.utils.VerifiableProperties) [2014-03-19 17:35:29,925] INFO Property group.id is overridden to mfirouz-vagrant-kafkamigration (kafka.utils.VerifiableProperties) [2014-03-19 17:35:29,926] WARN Property mirror.topics.whitelist is not valid (kafka.utils.VerifiableProperties) [2014-03-19 17:35:29,927] INFO Property zookeeper.connect is overridden to zookeeper-01.stag.visualdna.com:2181,zookeeper-02.stag.visualdna.com:2181, zookeeper-03.stag.visualdna.com:2181 (kafka.utils.VerifiableProperties) [2014-03-19 17:35:29,952] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Connecting to zookeeper instance at zookeeper-01.stag.visualdna.com:2181, zookeeper-02.stag.visualdna.com:2181,zookeeper-03.stag.visualdna.com:2181(kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:29,969] DEBUG Creating new ZookKeeper instance to connect to zookeeper-01.stag.visualdna.com:2181,zookeeper-02.stag.visualdna.com:2181 ,zookeeper-03.stag.visualdna.com:2181. (org.I0Itec.zkclient.ZkConnection) [2014-03-19 17:35:29,974] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread) [2014-03-19 17:35:29,982] INFO Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,982] INFO Client environment:host.name=precise64 (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,982] INFO Client environment:java.version=1.7.0_51 (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,982] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,983] INFO Client environment:java.home=/opt/java/jdk1.7.0_51/jre (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,983] INFO Client environment:java.class.path=:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../core/build/dependant-libs-2.8.0/*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../perf/build/libs//kafka-perf_2.8.0*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../clients/build/libs//kafka-clients*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../examples/build/libs//kafka-examples*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/jopt-simple-3.2.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/kafka_2.9.2-0.8.1.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/log4j-1.2.15.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/metrics-annotation-2.2.0.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/metrics-core-2.2.0.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/scala-library-2.9.2.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/slf4j-api-1.7.2.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/snappy-java-1.0.5.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/zkclient-0.3.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/zookeeper-3.3.4.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../core/build/libs/kafka_2.8.0*.jar (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,983] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,984] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,985] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,985] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,985] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,986] INFO Client environment:os.version=3.2.0-23-generic (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,986] INFO Client environment:user.name=vagrant (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,987] INFO Client environment:user.home=/home/vagrant (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,987] INFO Client environment:user.dir=/home/vagrant/Desktop (org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,990] INFO Initiating client connection, connectString= zookeeper-01.stag.visualdna.com:2181,zookeeper-02.stag.visualdna.com:2181, zookeeper-03.stag.visualdna.com:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@50e7a7f2(org.apache.zookeeper.ZooKeeper) [2014-03-19 17:35:29,993] DEBUG zookeeper.disableAutoWatchReset is false (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,027] DEBUG Awaiting connection to Zookeeper server (org.I0Itec.zkclient.ZkClient) [2014-03-19 17:35:30,027] INFO Opening socket connection to server zookeeper-03.stag.visualdna.com/10.200.10.63:2181(org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,027] DEBUG Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient) [2014-03-19 17:35:30,042] INFO Socket connection established to zookeeper-03.stag.visualdna.com/10.200.10.63:2181, initiating session (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,047] DEBUG Session establishment request sent on zookeeper-03.stag.visualdna.com/10.200.10.63:2181(org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,068] INFO Session establishment complete on server zookeeper-03.stag.visualdna.com/10.200.10.63:2181, sessionid = 0x3448da1f8f0644b, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,072] DEBUG Received event: WatchedEvent state:SyncConnected type:None path:null (org.I0Itec.zkclient.ZkClient) [2014-03-19 17:35:30,073] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient) [2014-03-19 17:35:30,073] DEBUG Leaving process event (org.I0Itec.zkclient.ZkClient) [2014-03-19 17:35:30,073] DEBUG State is SyncConnected (org.I0Itec.zkclient.ZkClient) SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. [2014-03-19 17:35:30,111] DEBUG Initializing task scheduler. (kafka.utils.KafkaScheduler) [2014-03-19 17:35:30,114] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], starting auto committer every 60000 ms (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,118] DEBUG Scheduling task kafka-consumer-autocommit with initial delay 60000 ms and period 60000 ms. (kafka.utils.KafkaScheduler) [2014-03-19 17:35:30,210] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 1,8 replyHeader:: 1,150361056246,0 request:: '/brokers/topics,F response:: v{'profileupdates_gsantoro,'prod_interactions,'datasync,'audiencemembership,'prod_profileupdates,'audienceactivity,'dataexport,'prod_useractivity,'dataexport-gsantoro,'browsingactivity,'tracking_events,'profileactivity,'prod_conversions,'prod_pageviews,'prod_datasync,'prod_dataexport,'profileupdates,'pageviews,'useractivity,'conversions,'prod_profileactivity,'interactions,'quizactivity} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,221] DEBUG profileupdates_gsantoro filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,222] DEBUG prod_interactions filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,222] DEBUG datasync filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,223] DEBUG audiencemembership filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,224] DEBUG prod_profileupdates filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,224] DEBUG audienceactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,225] DEBUG dataexport filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,225] DEBUG prod_useractivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,225] DEBUG dataexport-gsantoro filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,226] DEBUG browsingactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,226] DEBUG tracking_events filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,227] DEBUG profileactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,227] DEBUG prod_conversions allowed (kafka.consumer.Whitelist) [2014-03-19 17:35:30,229] DEBUG prod_pageviews filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,230] DEBUG prod_datasync filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,230] DEBUG prod_dataexport filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,231] DEBUG profileupdates filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,231] DEBUG pageviews filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,232] DEBUG useractivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,232] DEBUG conversions filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,233] DEBUG prod_profileactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,234] DEBUG interactions filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,234] DEBUG quizactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,252] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin registering consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce in ZK (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,409] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 2,1 replyHeader:: 2,150361056248,0 request:: '/consumers/mfirouz-vagrant-kafkamigration/ids/mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce,#7b2276657273696f6e223a312c22737562736372697074696f6e223a7b2270726f645f636f6e76657273696f6e73223a317d2c227061747465726e223a2277686974655f6c697374222c2274696d657374616d70223a2231333935323530353330323537227d,v{s{31,s{'world,'anyone}}},1 response:: '/consumers/mfirouz-vagrant-kafkamigration/ids/mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,410] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end registering consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce in ZK (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,427] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], starting watcher executor thread for consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,433] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 3,8 replyHeader:: 3,150361056250,0 request:: '/brokers/topics,F response:: v{'profileupdates_gsantoro,'prod_interactions,'datasync,'audiencemembership,'prod_profileupdates,'audienceactivity,'dataexport,'prod_useractivity,'dataexport-gsantoro,'browsingactivity,'tracking_events,'profileactivity,'prod_conversions,'prod_pageviews,'prod_datasync,'prod_dataexport,'profileupdates,'pageviews,'useractivity,'conversions,'prod_profileactivity,'interactions,'quizactivity} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,437] DEBUG profileupdates_gsantoro filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,437] DEBUG prod_interactions filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,437] DEBUG datasync filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,438] DEBUG audiencemembership filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,438] DEBUG prod_profileupdates filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,438] DEBUG audienceactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,438] DEBUG dataexport filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,439] DEBUG prod_useractivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,439] DEBUG dataexport-gsantoro filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,439] DEBUG browsingactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,439] DEBUG tracking_events filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,440] DEBUG profileactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,440] DEBUG prod_conversions allowed (kafka.consumer.Whitelist) [2014-03-19 17:35:30,440] DEBUG prod_pageviews filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,441] DEBUG prod_datasync filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,441] DEBUG prod_dataexport filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,441] DEBUG profileupdates filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,441] DEBUG pageviews filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,441] DEBUG useractivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,442] DEBUG conversions filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,442] DEBUG prod_profileactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,442] DEBUG interactions filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,442] DEBUG quizactivity filtered (kafka.consumer.Whitelist) [2014-03-19 17:35:30,491] DEBUG [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Adding topicThreadId (prod_conversions,mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce-0) and queue [] to topicThreadIdAndQueues data structure (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,499] DEBUG [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], adding topic prod_conversions and 1 streams to map. (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,517] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 4,3 replyHeader:: 4,150361056251,0 request:: '/consumers/mfirouz-vagrant-kafkamigration/ids,T response:: s{150358573750,150358573750,1395165000820,1395165000820,0,35,0,0,0,1,150361056248} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,525] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 5,8 replyHeader:: 5,150361056251,0 request:: '/consumers/mfirouz-vagrant-kafkamigration/ids,T response:: v{'mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,536] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 6,3 replyHeader:: 6,150361056251,0 request:: '/brokers/topics/prod_conversions,T response:: s{42963010923,42963010923,1363690601510,1363690601510,0,883,0,0,0,3,150357230474} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,536] DEBUG Subscribed data changes for /brokers/topics/prod_conversions (org.I0Itec.zkclient.ZkClient) [2014-03-19 17:35:30,542] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #0 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,550] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 7,8 replyHeader:: 7,150361056251,0 request:: '/brokers/ids,F response:: v{'3,'2,'1} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,563] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 8,4 replyHeader:: 8,150361056251,0 request:: '/brokers/ids/3,F response:: #626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:30,649] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], exception during rebalance (kafka.consumer.ZookeeperConsumerConnector) kafka.common.KafkaException: Failed to parse the broker info from zookeeper: bl-queue-s03.visualdna.com-1394796157928: bl-queue-s03.visualdna.com:9092 at kafka.cluster.Broker$.createBroker(Broker.scala:45) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504) at scala.collection.Iterator$class.foreach(Iterator.scala:772) at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573) at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) at scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615) at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402) at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not exist at kafka.cluster.Broker$.createBroker(Broker.scala:42) ... 20 more [2014-03-19 17:35:30,669] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #0 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,670] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Rebalancing attempt failed. Clearing the cache before the next rebalancing operation is triggered (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,674] INFO [ConsumerFetcherManager-1395250530084] Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:30,675] INFO [ConsumerFetcherManager-1395250530084] Stopping all fetchers (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:30,677] INFO [ConsumerFetcherManager-1395250530084] All connections stopped (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:30,679] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared all relevant queues for this fetcher (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,681] DEBUG Clearing the current data chunk for this consumer iterator (kafka.consumer.ConsumerIterator) [2014-03-19 17:35:30,682] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared the data chunks in all the consumer message iterators (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:30,683] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Committing all offsets after clearing the fetcher queues (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:32,603] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 41ms (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:32,687] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #1 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:32,693] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 9,8 replyHeader:: 9,150361056321,0 request:: '/brokers/ids,F response:: v{'3,'2,'1} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:32,712] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 10,4 replyHeader:: 10,150361056321,0 request:: '/brokers/ids/3,F response:: #626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:32,715] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], exception during rebalance (kafka.consumer.ZookeeperConsumerConnector) kafka.common.KafkaException: Failed to parse the broker info from zookeeper: bl-queue-s03.visualdna.com-1394796157928: bl-queue-s03.visualdna.com:9092 at kafka.cluster.Broker$.createBroker(Broker.scala:45) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504) at scala.collection.Iterator$class.foreach(Iterator.scala:772) at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573) at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) at scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615) at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402) at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not exist at kafka.cluster.Broker$.createBroker(Broker.scala:42) ... 20 more [2014-03-19 17:35:32,720] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #1 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:32,720] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Rebalancing attempt failed. Clearing the cache before the next rebalancing operation is triggered (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:32,721] INFO [ConsumerFetcherManager-1395250530084] Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:32,721] INFO [ConsumerFetcherManager-1395250530084] Stopping all fetchers (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:32,721] INFO [ConsumerFetcherManager-1395250530084] All connections stopped (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:32,722] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared all relevant queues for this fetcher (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:32,722] DEBUG Clearing the current data chunk for this consumer iterator (kafka.consumer.ConsumerIterator) [2014-03-19 17:35:32,722] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared the data chunks in all the consumer message iterators (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:32,723] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Committing all offsets after clearing the fetcher queues (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:34,705] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 6ms (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:34,724] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #2 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:34,732] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 11,8 replyHeader:: 11,150361056365,0 request:: '/brokers/ids,F response:: v{'3,'2,'1} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:34,739] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 12,4 replyHeader:: 12,150361056365,0 request:: '/brokers/ids/3,F response:: #626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:34,741] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], exception during rebalance (kafka.consumer.ZookeeperConsumerConnector) kafka.common.KafkaException: Failed to parse the broker info from zookeeper: bl-queue-s03.visualdna.com-1394796157928: bl-queue-s03.visualdna.com:9092 at kafka.cluster.Broker$.createBroker(Broker.scala:45) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504) at scala.collection.Iterator$class.foreach(Iterator.scala:772) at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573) at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) at scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615) at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402) at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not exist at kafka.cluster.Broker$.createBroker(Broker.scala:42) ... 20 more [2014-03-19 17:35:34,744] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #2 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:34,744] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Rebalancing attempt failed. Clearing the cache before the next rebalancing operation is triggered (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:34,745] INFO [ConsumerFetcherManager-1395250530084] Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:34,745] INFO [ConsumerFetcherManager-1395250530084] Stopping all fetchers (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:34,746] INFO [ConsumerFetcherManager-1395250530084] All connections stopped (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:34,746] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared all relevant queues for this fetcher (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:34,746] DEBUG Clearing the current data chunk for this consumer iterator (kafka.consumer.ConsumerIterator) [2014-03-19 17:35:34,747] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared the data chunks in all the consumer message iterators (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:34,747] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Committing all offsets after clearing the fetcher queues (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:36,744] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 6ms (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:36,749] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #3 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:36,762] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 13,8 replyHeader:: 13,150361056417,0 request:: '/brokers/ids,F response:: v{'3,'2,'1} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:36,782] DEBUG Reading reply sessionid:0x3448da1f8f0644b, packet:: clientPath:null serverPath:null finished:false header:: 14,4 replyHeader:: 14,150361056418,0 request:: '/brokers/ids/3,F response:: #626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985} (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:36,785] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], exception during rebalance (kafka.consumer.ZookeeperConsumerConnector) kafka.common.KafkaException: Failed to parse the broker info from zookeeper: bl-queue-s03.visualdna.com-1394796157928: bl-queue-s03.visualdna.com:9092 at kafka.cluster.Broker$.createBroker(Broker.scala:45) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506) at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504) at scala.collection.Iterator$class.foreach(Iterator.scala:772) at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573) at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) at scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615) at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:81) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402) at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not exist at kafka.cluster.Broker$.createBroker(Broker.scala:42) ... 20 more [2014-03-19 17:35:36,788] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end rebalancing consumer mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #3 (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:36,789] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Rebalancing attempt failed. Clearing the cache before the next rebalancing operation is triggered (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:36,789] INFO [ConsumerFetcherManager-1395250530084] Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:36,789] INFO [ConsumerFetcherManager-1395250530084] Stopping all fetchers (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:36,789] INFO [ConsumerFetcherManager-1395250530084] All connections stopped (kafka.consumer.ConsumerFetcherManager) [2014-03-19 17:35:36,790] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared all relevant queues for this fetcher (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:36,790] DEBUG Clearing the current data chunk for this consumer iterator (kafka.consumer.ConsumerIterator) [2014-03-19 17:35:36,790] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared the data chunks in all the consumer message iterators (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:36,790] INFO [mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Committing all offsets after clearing the fetcher queues (kafka.consumer.ZookeeperConsumerConnector) [2014-03-19 17:35:38,779] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 5ms (org.apache.zookeeper.ClientCnxn) Kafka migration tool failed due to: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) Caused by: kafka.common.ConsumerRebalanceFailedException: mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce can't rebalance after 4 retries at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) ... 5 more [2014-03-19 17:35:38,799] ERROR Kafka migration tool failed: (kafka.tools.KafkaMigrationTool) java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) Caused by: kafka.common.ConsumerRebalanceFailedException: mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce can't rebalance after 4 retries at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) ... 5 more [2014-03-19 17:35:40,783] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 7ms (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:42,786] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 7ms (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:44,787] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 6ms (org.apache.zookeeper.ClientCnxn) Thanks, Mo. On 19 March 2014 14:07, Mo Firouz <mo.fir...@visualdna.com> wrote: > Hello. > > I am trying to migrate from Kafka 0.7 to 0.8, and the first stage of this > is to use the Migration Tool. > > I have setup a local Kafka 0.8 cluster consisting of 3 nodes on my local > Vagrant (along with a zookeeper running locally). The Migration Tool > Producer will connect to these three nodes in order to replicate the data. > The Migration Tool Consumer(s) will connect to my Staging boxes to consume > Kafka 0.7 events. > > However, upon running the Migration Tool, I'll get the following error: > > *vagrant@precise64:~/Desktop$ >> ~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh >> kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar >> --zkclient.01.jar zkclient-0.2.0.jar --num.producers 16 >> --producer.config=/home/vagrant/Deskt/producerConfig.properties >> --consumer.config=/home/vagrant/Desktop/consumerConfig.properties >> --whitelist "prod_conversions"* > > [2014-03-19 13:14:38,126] WARN Property mirror.topics.whitelist is not >> valid (kafka.utils.VerifiableProperties) >> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". >> SLF4J: Defaulting to no-operation (NOP) logger implementation >> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for >> further details. >> Kafka migration tool failed due to: >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) >> Caused by: kafka.common.ConsumerRebalanceFailedException: >> mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't >> rebalance after 4 retries >> at >> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) >> at >> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) >> at >> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) >> at >> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) >> ... 5 more >> [2014-03-19 13:14:47,082] ERROR Kafka migration tool failed: >> (kafka.tools.KafkaMigrationTool) >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) >> Caused by: kafka.common.ConsumerRebalanceFailedException: >> mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't >> rebalance after 4 retries >> at >> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) >> at >> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) >> at >> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756) >> at >> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96) >> at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) >> ... 5 more > > > My 3 *Kafka server* configurations are as follows: > > *cat kafka-server1.properties * >> zookeeper.connect=localhost:2181 >> zookeeper.connection.timeout.ms=1000000 >> broker.id=1 >> port=9092 >> log.dir=/tmp/kafka-logs-1 > > > >> >> * cat kafka-server2.properties * >> zookeeper.connect=localhost:2181 >> zookeeper.connection.timeout.ms=1000000 >> broker.id=1 >> port=9093 >> log.dir=/tmp/kafka-logs-2 > > > >> *cat kafka-server3.properties * >> zookeeper.connect=localhost:2181 >> zookeeper.connection.timeout.ms=1000000 >> broker.id=1 >> port=9094 >> log.dir=/tmp/kafka-logs-3 > > > My *consumer* configuration is: > > *cat consumerConfig.properties * >> group.id = mfirouz-vagrant-kafkamigration >> zookeeper.connect = zk-01.s.visualdna.com:2181,zk-02.s.visualdna.com:2181 >> ,zk-03.s.visualdna.com:2181 >> mirror.topics.whitelist = "prod_conversions" > > > And finally, my *producer* configuration is: > > *cat producerConfig.properties* >> zk.connect = localhost:2181 >> whitelist = "prod_conversions" >> producer.type = "sync" >> request.required.acks = 1 >> metadata.broker.list = localhost:9092,localhost:9093,localhost:9094 > > > I was wondering if anyone had any insight about this and how to fix it? > > Thanks a lot in advance > Mo. >
