Hi Jun,

Thanks for the reply.

Please find below the DEBUG output of the log.

*vagrant@precise64:~/Desktop$
~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh
kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar
--zkclient.01.jar zkclient-0.2.0.jar --num.producers 16
--producer.config=/home/vagrant/Desktop/producerConfig.properties
--consumer.config=/home/vagrant/Desktop/consumerConfig.properties
--whitelist "prod_conversions"*
[2014-03-19 17:35:29,856] INFO Verifying properties
(kafka.utils.VerifiableProperties)
[2014-03-19 17:35:29,925] INFO Property group.id is overridden to
mfirouz-vagrant-kafkamigration (kafka.utils.VerifiableProperties)
[2014-03-19 17:35:29,926] WARN Property mirror.topics.whitelist is not
valid (kafka.utils.VerifiableProperties)
[2014-03-19 17:35:29,927] INFO Property zookeeper.connect is overridden to
zookeeper-01.stag.visualdna.com:2181,zookeeper-02.stag.visualdna.com:2181,
zookeeper-03.stag.visualdna.com:2181 (kafka.utils.VerifiableProperties)
[2014-03-19 17:35:29,952] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Connecting to zookeeper instance at zookeeper-01.stag.visualdna.com:2181,
zookeeper-02.stag.visualdna.com:2181,zookeeper-03.stag.visualdna.com:2181(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:29,969] DEBUG Creating new ZookKeeper instance to connect
to zookeeper-01.stag.visualdna.com:2181,zookeeper-02.stag.visualdna.com:2181
,zookeeper-03.stag.visualdna.com:2181. (org.I0Itec.zkclient.ZkConnection)
[2014-03-19 17:35:29,974] INFO Starting ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
[2014-03-19 17:35:29,982] INFO Client
environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,982] INFO Client environment:host.name=precise64
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,982] INFO Client environment:java.version=1.7.0_51
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,982] INFO Client environment:java.vendor=Oracle
Corporation (org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,983] INFO Client
environment:java.home=/opt/java/jdk1.7.0_51/jre
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,983] INFO Client
environment:java.class.path=:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../core/build/dependant-libs-2.8.0/*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../perf/build/libs//kafka-perf_2.8.0*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../clients/build/libs//kafka-clients*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../examples/build/libs//kafka-examples*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/jopt-simple-3.2.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/kafka_2.9.2-0.8.1.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/log4j-1.2.15.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/metrics-annotation-2.2.0.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/metrics-core-2.2.0.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/scala-library-2.9.2.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/slf4j-api-1.7.2.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/snappy-java-1.0.5.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/zkclient-0.3.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../libs/zookeeper-3.3.4.jar:/home/vagrant/Desktop/kafka_2.9.2-0.8.1/bin/../core/build/libs/kafka_2.8.0*.jar
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,983] INFO Client
environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,984] INFO Client environment:java.io.tmpdir=/tmp
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,985] INFO Client environment:java.compiler=<NA>
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,985] INFO Client environment:os.name=Linux
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,985] INFO Client environment:os.arch=amd64
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,986] INFO Client
environment:os.version=3.2.0-23-generic (org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,986] INFO Client environment:user.name=vagrant
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,987] INFO Client environment:user.home=/home/vagrant
(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,987] INFO Client
environment:user.dir=/home/vagrant/Desktop (org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,990] INFO Initiating client connection, connectString=
zookeeper-01.stag.visualdna.com:2181,zookeeper-02.stag.visualdna.com:2181,
zookeeper-03.stag.visualdna.com:2181 sessionTimeout=6000
watcher=org.I0Itec.zkclient.ZkClient@50e7a7f2(org.apache.zookeeper.ZooKeeper)
[2014-03-19 17:35:29,993] DEBUG zookeeper.disableAutoWatchReset is false
(org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,027] DEBUG Awaiting connection to Zookeeper server
(org.I0Itec.zkclient.ZkClient)
[2014-03-19 17:35:30,027] INFO Opening socket connection to server
zookeeper-03.stag.visualdna.com/10.200.10.63:2181(org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,027] DEBUG Waiting for keeper state SyncConnected
(org.I0Itec.zkclient.ZkClient)
[2014-03-19 17:35:30,042] INFO Socket connection established to
zookeeper-03.stag.visualdna.com/10.200.10.63:2181, initiating session
(org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,047] DEBUG Session establishment request sent on
zookeeper-03.stag.visualdna.com/10.200.10.63:2181(org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,068] INFO Session establishment complete on server
zookeeper-03.stag.visualdna.com/10.200.10.63:2181, sessionid =
0x3448da1f8f0644b, negotiated timeout = 6000
(org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,072] DEBUG Received event: WatchedEvent
state:SyncConnected type:None path:null (org.I0Itec.zkclient.ZkClient)
[2014-03-19 17:35:30,073] INFO zookeeper state changed (SyncConnected)
(org.I0Itec.zkclient.ZkClient)
[2014-03-19 17:35:30,073] DEBUG Leaving process event
(org.I0Itec.zkclient.ZkClient)
[2014-03-19 17:35:30,073] DEBUG State is SyncConnected
(org.I0Itec.zkclient.ZkClient)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
[2014-03-19 17:35:30,111] DEBUG Initializing task scheduler.
(kafka.utils.KafkaScheduler)
[2014-03-19 17:35:30,114] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], starting
auto committer every 60000 ms (kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,118] DEBUG Scheduling task kafka-consumer-autocommit
with initial delay 60000 ms and period 60000 ms.
(kafka.utils.KafkaScheduler)
[2014-03-19 17:35:30,210] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 1,8
 replyHeader:: 1,150361056246,0  request:: '/brokers/topics,F  response::
v{'profileupdates_gsantoro,'prod_interactions,'datasync,'audiencemembership,'prod_profileupdates,'audienceactivity,'dataexport,'prod_useractivity,'dataexport-gsantoro,'browsingactivity,'tracking_events,'profileactivity,'prod_conversions,'prod_pageviews,'prod_datasync,'prod_dataexport,'profileupdates,'pageviews,'useractivity,'conversions,'prod_profileactivity,'interactions,'quizactivity}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,221] DEBUG profileupdates_gsantoro filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,222] DEBUG prod_interactions filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,222] DEBUG datasync filtered (kafka.consumer.Whitelist)
[2014-03-19 17:35:30,223] DEBUG audiencemembership filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,224] DEBUG prod_profileupdates filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,224] DEBUG audienceactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,225] DEBUG dataexport filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,225] DEBUG prod_useractivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,225] DEBUG dataexport-gsantoro filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,226] DEBUG browsingactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,226] DEBUG tracking_events filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,227] DEBUG profileactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,227] DEBUG prod_conversions allowed
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,229] DEBUG prod_pageviews filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,230] DEBUG prod_datasync filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,230] DEBUG prod_dataexport filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,231] DEBUG profileupdates filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,231] DEBUG pageviews filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,232] DEBUG useractivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,232] DEBUG conversions filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,233] DEBUG prod_profileactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,234] DEBUG interactions filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,234] DEBUG quizactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,252] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin
registering consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce in ZK
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,409] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 2,1
 replyHeader:: 2,150361056248,0  request::
'/consumers/mfirouz-vagrant-kafkamigration/ids/mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce,#7b2276657273696f6e223a312c22737562736372697074696f6e223a7b2270726f645f636f6e76657273696f6e73223a317d2c227061747465726e223a2277686974655f6c697374222c2274696d657374616d70223a2231333935323530353330323537227d,v{s{31,s{'world,'anyone}}},1
 response::
'/consumers/mfirouz-vagrant-kafkamigration/ids/mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,410] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end
registering consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce in ZK
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,427] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], starting
watcher executor thread for consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,433] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 3,8
 replyHeader:: 3,150361056250,0  request:: '/brokers/topics,F  response::
v{'profileupdates_gsantoro,'prod_interactions,'datasync,'audiencemembership,'prod_profileupdates,'audienceactivity,'dataexport,'prod_useractivity,'dataexport-gsantoro,'browsingactivity,'tracking_events,'profileactivity,'prod_conversions,'prod_pageviews,'prod_datasync,'prod_dataexport,'profileupdates,'pageviews,'useractivity,'conversions,'prod_profileactivity,'interactions,'quizactivity}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,437] DEBUG profileupdates_gsantoro filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,437] DEBUG prod_interactions filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,437] DEBUG datasync filtered (kafka.consumer.Whitelist)
[2014-03-19 17:35:30,438] DEBUG audiencemembership filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,438] DEBUG prod_profileupdates filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,438] DEBUG audienceactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,438] DEBUG dataexport filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,439] DEBUG prod_useractivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,439] DEBUG dataexport-gsantoro filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,439] DEBUG browsingactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,439] DEBUG tracking_events filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,440] DEBUG profileactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,440] DEBUG prod_conversions allowed
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,440] DEBUG prod_pageviews filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,441] DEBUG prod_datasync filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,441] DEBUG prod_dataexport filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,441] DEBUG profileupdates filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,441] DEBUG pageviews filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,441] DEBUG useractivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,442] DEBUG conversions filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,442] DEBUG prod_profileactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,442] DEBUG interactions filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,442] DEBUG quizactivity filtered
(kafka.consumer.Whitelist)
[2014-03-19 17:35:30,491] DEBUG
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Adding
topicThreadId
(prod_conversions,mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce-0)
and queue [] to topicThreadIdAndQueues data structure
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,499] DEBUG
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], adding
topic prod_conversions and 1 streams to map.
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,517] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 4,3
 replyHeader:: 4,150361056251,0  request::
'/consumers/mfirouz-vagrant-kafkamigration/ids,T  response::
s{150358573750,150358573750,1395165000820,1395165000820,0,35,0,0,0,1,150361056248}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,525] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 5,8
 replyHeader:: 5,150361056251,0  request::
'/consumers/mfirouz-vagrant-kafkamigration/ids,T  response::
v{'mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,536] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 6,3
 replyHeader:: 6,150361056251,0  request::
'/brokers/topics/prod_conversions,T  response::
s{42963010923,42963010923,1363690601510,1363690601510,0,883,0,0,0,3,150357230474}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,536] DEBUG Subscribed data changes for
/brokers/topics/prod_conversions (org.I0Itec.zkclient.ZkClient)
[2014-03-19 17:35:30,542] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #0
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,550] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 7,8
 replyHeader:: 7,150361056251,0  request:: '/brokers/ids,F  response::
v{'3,'2,'1}  (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,563] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 8,4
 replyHeader:: 8,150361056251,0  request:: '/brokers/ids/3,F  response::
#626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:30,649] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
exception during rebalance  (kafka.consumer.ZookeeperConsumerConnector)
kafka.common.KafkaException: Failed to parse the broker info from
zookeeper: bl-queue-s03.visualdna.com-1394796157928:
bl-queue-s03.visualdna.com:9092
at kafka.cluster.Broker$.createBroker(Broker.scala:45)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at
scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)
at
scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615)
at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at
kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not
exist
at kafka.cluster.Broker$.createBroker(Broker.scala:42)
... 20 more
[2014-03-19 17:35:30,669] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #0
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,670] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Rebalancing attempt failed. Clearing the cache before the next rebalancing
operation is triggered (kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,674] INFO [ConsumerFetcherManager-1395250530084]
Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:30,675] INFO [ConsumerFetcherManager-1395250530084]
Stopping all fetchers (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:30,677] INFO [ConsumerFetcherManager-1395250530084] All
connections stopped (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:30,679] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
all relevant queues for this fetcher
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,681] DEBUG Clearing the current data chunk for this
consumer iterator (kafka.consumer.ConsumerIterator)
[2014-03-19 17:35:30,682] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
the data chunks in all the consumer message iterators
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:30,683] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Committing all offsets after clearing the fetcher queues
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:32,603] DEBUG Got ping response for sessionid:
0x3448da1f8f0644b after 41ms (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:32,687] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #1
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:32,693] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 9,8
 replyHeader:: 9,150361056321,0  request:: '/brokers/ids,F  response::
v{'3,'2,'1}  (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:32,712] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 10,4
 replyHeader:: 10,150361056321,0  request:: '/brokers/ids/3,F  response::
#626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:32,715] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
exception during rebalance  (kafka.consumer.ZookeeperConsumerConnector)
kafka.common.KafkaException: Failed to parse the broker info from
zookeeper: bl-queue-s03.visualdna.com-1394796157928:
bl-queue-s03.visualdna.com:9092
at kafka.cluster.Broker$.createBroker(Broker.scala:45)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at
scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)
at
scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615)
at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at
kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not
exist
at kafka.cluster.Broker$.createBroker(Broker.scala:42)
... 20 more
[2014-03-19 17:35:32,720] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #1
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:32,720] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Rebalancing attempt failed. Clearing the cache before the next rebalancing
operation is triggered (kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:32,721] INFO [ConsumerFetcherManager-1395250530084]
Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:32,721] INFO [ConsumerFetcherManager-1395250530084]
Stopping all fetchers (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:32,721] INFO [ConsumerFetcherManager-1395250530084] All
connections stopped (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:32,722] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
all relevant queues for this fetcher
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:32,722] DEBUG Clearing the current data chunk for this
consumer iterator (kafka.consumer.ConsumerIterator)
[2014-03-19 17:35:32,722] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
the data chunks in all the consumer message iterators
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:32,723] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Committing all offsets after clearing the fetcher queues
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:34,705] DEBUG Got ping response for sessionid:
0x3448da1f8f0644b after 6ms (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:34,724] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #2
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:34,732] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 11,8
 replyHeader:: 11,150361056365,0  request:: '/brokers/ids,F  response::
v{'3,'2,'1}  (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:34,739] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 12,4
 replyHeader:: 12,150361056365,0  request:: '/brokers/ids/3,F  response::
#626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:34,741] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
exception during rebalance  (kafka.consumer.ZookeeperConsumerConnector)
kafka.common.KafkaException: Failed to parse the broker info from
zookeeper: bl-queue-s03.visualdna.com-1394796157928:
bl-queue-s03.visualdna.com:9092
at kafka.cluster.Broker$.createBroker(Broker.scala:45)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at
scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)
at
scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615)
at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at
kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not
exist
at kafka.cluster.Broker$.createBroker(Broker.scala:42)
... 20 more
[2014-03-19 17:35:34,744] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #2
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:34,744] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Rebalancing attempt failed. Clearing the cache before the next rebalancing
operation is triggered (kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:34,745] INFO [ConsumerFetcherManager-1395250530084]
Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:34,745] INFO [ConsumerFetcherManager-1395250530084]
Stopping all fetchers (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:34,746] INFO [ConsumerFetcherManager-1395250530084] All
connections stopped (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:34,746] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
all relevant queues for this fetcher
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:34,746] DEBUG Clearing the current data chunk for this
consumer iterator (kafka.consumer.ConsumerIterator)
[2014-03-19 17:35:34,747] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
the data chunks in all the consumer message iterators
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:34,747] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Committing all offsets after clearing the fetcher queues
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:36,744] DEBUG Got ping response for sessionid:
0x3448da1f8f0644b after 6ms (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:36,749] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], begin
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #3
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:36,762] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 13,8
 replyHeader:: 13,150361056417,0  request:: '/brokers/ids,F  response::
v{'3,'2,'1}  (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:36,782] DEBUG Reading reply sessionid:0x3448da1f8f0644b,
packet:: clientPath:null serverPath:null finished:false header:: 14,4
 replyHeader:: 14,150361056418,0  request:: '/brokers/ids/3,F  response::
#626c2d71756575652d7330332e76697375616c646e612e636f6d2d313339343739363135373932383a626c2d71756575652d7330332e76697375616c646e612e636f6d3a39303932,s{150346957985,150346957985,1394796158123,1394796158123,0,0,0,163411213060484448,72,0,150346957985}
 (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:36,785] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
exception during rebalance  (kafka.consumer.ZookeeperConsumerConnector)
kafka.common.KafkaException: Failed to parse the broker info from
zookeeper: bl-queue-s03.visualdna.com-1394796157928:
bl-queue-s03.visualdna.com:9092
at kafka.cluster.Broker$.createBroker(Broker.scala:45)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:506)
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at
scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)
at
scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615)
at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:81)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at
kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
Caused by: kafka.common.BrokerNotAvailableException: Broker id 3 does not
exist
at kafka.cluster.Broker$.createBroker(Broker.scala:42)
... 20 more
[2014-03-19 17:35:36,788] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], end
rebalancing consumer
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce try #3
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:36,789] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Rebalancing attempt failed. Clearing the cache before the next rebalancing
operation is triggered (kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:36,789] INFO [ConsumerFetcherManager-1395250530084]
Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:36,789] INFO [ConsumerFetcherManager-1395250530084]
Stopping all fetchers (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:36,789] INFO [ConsumerFetcherManager-1395250530084] All
connections stopped (kafka.consumer.ConsumerFetcherManager)
[2014-03-19 17:35:36,790] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
all relevant queues for this fetcher
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:36,790] DEBUG Clearing the current data chunk for this
consumer iterator (kafka.consumer.ConsumerIterator)
[2014-03-19 17:35:36,790] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce], Cleared
the data chunks in all the consumer message iterators
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:36,790] INFO
[mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce],
Committing all offsets after clearing the fetcher queues
(kafka.consumer.ZookeeperConsumerConnector)
[2014-03-19 17:35:38,779] DEBUG Got ping response for sessionid:
0x3448da1f8f0644b after 5ms (org.apache.zookeeper.ClientCnxn)
Kafka migration tool failed due to:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
Caused by: kafka.common.ConsumerRebalanceFailedException:
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce can't
rebalance after 4 retries
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at
kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
... 5 more

[2014-03-19 17:35:38,799] ERROR Kafka migration tool failed:
 (kafka.tools.KafkaMigrationTool)
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
Caused by: kafka.common.ConsumerRebalanceFailedException:
mfirouz-vagrant-kafkamigration_precise64-1395250529941-562df2ce can't
rebalance after 4 retries
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
at
kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
... 5 more
[2014-03-19 17:35:40,783] DEBUG Got ping response for sessionid:
0x3448da1f8f0644b after 7ms (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:42,786] DEBUG Got ping response for sessionid:
0x3448da1f8f0644b after 7ms (org.apache.zookeeper.ClientCnxn)
[2014-03-19 17:35:44,787] DEBUG Got ping response for sessionid:
0x3448da1f8f0644b after 6ms (org.apache.zookeeper.ClientCnxn)

Thanks,
Mo.

On 19 March 2014 14:07, Mo Firouz <mo.fir...@visualdna.com> wrote:

> Hello.
>
> I am trying to migrate from Kafka 0.7 to 0.8, and the first stage of this
> is to use the Migration Tool.
>
> I have setup a local Kafka 0.8 cluster consisting of 3 nodes on my local
> Vagrant (along with a zookeeper running locally). The Migration Tool
> Producer will connect to these three nodes in order to replicate the data.
> The Migration Tool Consumer(s) will connect to my Staging boxes to consume
> Kafka 0.7 events.
>
> However, upon running the Migration Tool, I'll get the following error:
>
> *vagrant@precise64:~/Desktop$
>> ~/Desktop/kafka_2.9.2-0.8.1/bin/kafka-run-class.sh
>> kafka.tools.KafkaMigrationTool --kafka.07.jar kafka-0.7.19.jar
>> --zkclient.01.jar zkclient-0.2.0.jar --num.producers 16
>> --producer.config=/home/vagrant/Deskt/producerConfig.properties
>> --consumer.config=/home/vagrant/Desktop/consumerConfig.properties
>> --whitelist "prod_conversions"*
>
> [2014-03-19 13:14:38,126] WARN Property mirror.topics.whitelist is not
>> valid (kafka.utils.VerifiableProperties)
>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>> SLF4J: Defaulting to no-operation (NOP) logger implementation
>> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
>> further details.
>> Kafka migration tool failed due to:
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
>> Caused by: kafka.common.ConsumerRebalanceFailedException:
>> mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't
>> rebalance after 4 retries
>> at
>> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432)
>> at
>> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
>> at
>> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
>> at
>> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
>> at
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
>> at
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
>> ... 5 more
>> [2014-03-19 13:14:47,082] ERROR Kafka migration tool failed:
>>  (kafka.tools.KafkaMigrationTool)
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217)
>> Caused by: kafka.common.ConsumerRebalanceFailedException:
>> mfirouz-vagrant-kafkamigration_precise64-1395234878153-3d6d9aaa can't
>> rebalance after 4 retries
>> at
>> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432)
>> at
>> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
>> at
>> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:756)
>> at
>> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:145)
>> at
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:96)
>> at
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100)
>> ... 5 more
>
>
> My 3 *Kafka server* configurations are as follows:
>
> *cat kafka-server1.properties *
>> zookeeper.connect=localhost:2181
>> zookeeper.connection.timeout.ms=1000000
>> broker.id=1
>> port=9092
>> log.dir=/tmp/kafka-logs-1
>
>
>
>>
>> * cat kafka-server2.properties *
>> zookeeper.connect=localhost:2181
>> zookeeper.connection.timeout.ms=1000000
>> broker.id=1
>> port=9093
>> log.dir=/tmp/kafka-logs-2
>
>
>
>> *cat kafka-server3.properties *
>> zookeeper.connect=localhost:2181
>> zookeeper.connection.timeout.ms=1000000
>> broker.id=1
>> port=9094
>> log.dir=/tmp/kafka-logs-3
>
>
> My *consumer* configuration is:
>
> *cat consumerConfig.properties *
>> group.id = mfirouz-vagrant-kafkamigration
>> zookeeper.connect = zk-01.s.visualdna.com:2181,zk-02.s.visualdna.com:2181
>> ,zk-03.s.visualdna.com:2181
>> mirror.topics.whitelist = "prod_conversions"
>
>
> And finally, my *producer* configuration is:
>
> *cat producerConfig.properties*
>> zk.connect = localhost:2181
>> whitelist = "prod_conversions"
>> producer.type = "sync"
>> request.required.acks = 1
>> metadata.broker.list = localhost:9092,localhost:9093,localhost:9094
>
>
> I was wondering if anyone had any insight about this and how to fix it?
>
> Thanks a lot in advance
> Mo.
>

Reply via email to