Re: Cannot connect to Kafka from outside of EC2

Guozhang Wang Tue, 11 Nov 2014 10:20:12 -0800

Hi Sameer,

I think apache mailing list has blocked your attachment. If it is too long
to include in the email body could you paste it somewhere and give me the
link?


Guozhang

On Tue, Nov 11, 2014 at 10:01 AM, Sameer Yami <sy...@ailive.net> wrote:

> Hi Guozhang,
>
> I was wondering if you found anything wrong in the logs/
>
> thanks
>
>
> On Fri, Nov 7, 2014 at 4:19 PM, Sameer Yami <sy...@ailive.net> wrote:
>
> > Hi Guozhang,
> >
> > Attached are the two logs with debug enabled.
> >
> > Thanks!
> >
> > On Fri, Nov 7, 2014 at 2:09 PM, Sameer Yami <sy...@ailive.net> wrote:
> >
> >> The version is kafka_2.10-0.8.1.1. It is not the latest trunk.
> >> Will try enabling debug version.
> >>
> >> thanks
> >>
> >>
> >> On Thu, Nov 6, 2014 at 9:37 PM, Guozhang Wang <wangg...@gmail.com>
> wrote:
> >>
> >>> Sameer,
> >>>
> >>> The server logs do not contain any non-INFO logs, which is a bit wired.
> >>> Did
> >>> you deploy the current trunk of Kafka? Also could you enable DEBUG
> level
> >>> logging on Kafka brokers?
> >>>
> >>> Guozhang
> >>>
> >>> On Wed, Nov 5, 2014 at 3:50 PM, Sameer Yami <sy...@ailive.net> wrote:
> >>>
> >>> > The server.log was taken separately.
> >>> > We ran the test again and the server and producer logs are below (to
> >>> get
> >>> > same timings).
> >>> >
> >>> >
> >>> > Thanks!
> >>> >
> >>> >
> >>> >
> >>>
> ------------------------------------------------------------------------------------
> >>> >
> >>> >
> >>> >
> >>> > Producer Logs -
> >>> >
> >>> >
> >>> > 2014-11-05 23:38:58,693
> >>> > Thread-3-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680002 after 0ms
> >>> > 2014-11-05 23:39:00,695
> >>> > Thread-3-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680002 after 0ms
> >>> > 2014-11-05 23:39:02,696
> >>> > Thread-3-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680002 after 0ms
> >>> > 2014-11-05 23:39:02,828 pool-13-thread-2   INFO
> >>> > kafka.utils.VerifiableProperties-68: Verifying properties
> >>> > 2014-11-05 23:39:02,829 pool-13-thread-2   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property
> auto.commit.interval.ms
> >>> is
> >>> > overridden to 1000
> >>> > 2014-11-05 23:39:02,829 pool-13-thread-2   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property auto.offset.reset is
> >>> > overridden to smallest
> >>> > 2014-11-05 23:39:02,829 pool-13-thread-2   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property consumer.timeout.ms is
> >>> > overridden to 10
> >>> > 2014-11-05 23:39:02,829 pool-13-thread-2   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property group.id is overridden
> >>> to
> >>> > TestCheck
> >>> > 2014-11-05 23:39:02,830 pool-13-thread-2   WARN
> >>> > kafka.utils.VerifiableProperties-83: Property serializer.class is not
> >>> valid
> >>> > 2014-11-05 23:39:02,830 pool-13-thread-2   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property zookeeper.connect is
> >>> > overridden to 172.31.25.198:2181
> >>> > 2014-11-05 23:39:02,831 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Connecting to
> >>> > zookeeper instance at 172.31.25.198:2181
> >>> > 2014-11-05 23:39:02,831 pool-13-thread-2  DEBUG
> >>> > org.I0Itec.zkclient.ZkConnection-63: Creating new ZookKeeper instance
> >>> to
> >>> > connect to 172.31.25.198:2181.
> >>> > 2014-11-05 23:39:02,831 pool-13-thread-2   INFO
> >>> > org.apache.zookeeper.ZooKeeper-379: Initiating client connection,
> >>> > connectString=172.31.25.198:2181 sessionTimeout=6000
> >>> > watcher=org.I0Itec.zkclient.ZkClient@3903b165
> >>> > 2014-11-05 23:39:02,831 ZkClient-EventThread-29-172.31.25.198:2181
> >>>  INFO
> >>> > org.I0Itec.zkclient.ZkEventThread-64: Starting ZkClient event thread.
> >>> > 2014-11-05 23:39:02,831 pool-13-thread-1   INFO
> >>> > kafka.utils.VerifiableProperties-68: Verifying properties
> >>> > 2014-11-05 23:39:02,836 pool-13-thread-2-SendThread()   INFO
> >>> > org.apache.zookeeper.ClientCnxn-1061: Opening socket connection to
> >>> server /
> >>> > 172.31.25.198:2181
> >>> > 2014-11-05 23:39:02,836 pool-13-thread-1   WARN
> >>> > kafka.utils.VerifiableProperties-83: Property batch.size is not valid
> >>> > 2014-11-05 23:39:02,832 pool-13-thread-2  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-878: Awaiting connection to Zookeeper
> >>> server
> >>> > 2014-11-05 23:39:02,836 pool-13-thread-1   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property
> message.send.max.retries
> >>> is
> >>> > overridden to 10
> >>> > 2014-11-05 23:39:02,836 pool-13-thread-2  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-628: Waiting for keeper state
> >>> SyncConnected
> >>> > 2014-11-05 23:39:02,837 pool-13-thread-1   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property metadata.broker.list is
> >>> > overridden to 172.31.25.198:9092
> >>> > 2014-11-05 23:39:02,837 pool-13-thread-1   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property retry.backoff.ms is
> >>> > overridden to 1000
> >>> > 2014-11-05 23:39:02,837 pool-13-thread-1   INFO
> >>> > kafka.utils.VerifiableProperties-68: Property serializer.class is
> >>> > overridden to kafka.serializer.StringEncoder
> >>> > 2014-11-05 23:39:02,837
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > INFO org.apache.zookeeper.ClientCnxn-950: Socket connection
> >>> established to
> >>> > ip-172-31-25-198.us-west-1.compute.internal/172.31.25.198:2181,
> >>> initiating
> >>> > session
> >>> > 2014-11-05 23:39:02,838
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-999: Session establishment
> >>> request
> >>> > sent on ip-172-31-25-198.us-west-1.compute.internal/
> 172.31.25.198:2181
> >>> > 2014-11-05 23:39:02,837 pool-13-thread-1   WARN
> >>> > kafka.utils.VerifiableProperties-83: Property
> zk.connectiontimeout.ms
> >>> is
> >>> > not valid
> >>> > 2014-11-05 23:39:02,841
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > INFO org.apache.zookeeper.ClientCnxn-739: Session establishment
> >>> complete on
> >>> > server ip-172-31-25-198.us-west-1.compute.internal/
> 172.31.25.198:2181,
> >>> > sessionid = 0x1498251e8680003, negotiated timeout = 6000
> >>> > 2014-11-05 23:39:02,841 pool-13-thread-2-EventThread  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-351: Received event: WatchedEvent
> >>> > state:SyncConnected type:None path:null
> >>> > 2014-11-05 23:39:02,841 pool-13-thread-2-EventThread   INFO
> >>> > org.I0Itec.zkclient.ZkClient-449: zookeeper state changed
> >>> (SyncConnected)
> >>> > 2014-11-05 23:39:02,841 pool-13-thread-2-EventThread  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-395: Leaving process event
> >>> > 2014-11-05 23:39:02,841 pool-13-thread-2  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-638: State is SyncConnected
> >>> > 2014-11-05 23:39:02,842 pool-13-thread-2  DEBUG
> >>> > kafka.utils.KafkaScheduler-52: Initializing task scheduler.
> >>> > 2014-11-05 23:39:02,842 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], starting auto
> >>> > committer every 1000 ms
> >>> > 2014-11-05 23:39:02,842 pool-13-thread-2  DEBUG
> >>> > kafka.utils.KafkaScheduler-52: Scheduling task
> >>> kafka-consumer-autocommit
> >>> > with initial delay 1000 ms and period 1000 ms.
> >>> > 2014-11-05 23:39:02,843 pool-13-thread-2  DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], entering consume
> >>> > 2014-11-05 23:39:02,843 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], begin
> registering
> >>> > consumer TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 in ZK
> >>> > 2014-11-05 23:39:02,846
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 1,1  replyHeader:: 1,29,-101  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/ids/TestCheck_ip-172-31-25-198-1415230742830-f3dfc362,#7b2276657273696f6e223a312c22737562736372697074696f6e223a7b22486561727462656174223a317d2c227061747465726e223a22737461746963222c2274696d657374616d70223a2231343135323330373432383433227d,v{s{31,s{'world,'anyone}}},1
> >>> > response::
> >>> > 2014-11-05 23:39:02,854
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 2,1  replyHeader:: 2,30,-101  request::
> >>> > '/consumers/TestCheck/ids,,v{s{31,s{'world,'anyone}}},0  response::
> >>> > 2014-11-05 23:39:02,858
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 3,1  replyHeader:: 3,31,0  request::
> >>> > '/consumers/TestCheck,,v{s{31,s{'world,'anyone}}},0  response::
> >>> > '/consumers/TestCheck
> >>> > 2014-11-05 23:39:02,861
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 4,1  replyHeader:: 4,32,0  request::
> >>> > '/consumers/TestCheck/ids,,v{s{31,s{'world,'anyone}}},0  response::
> >>> > '/consumers/TestCheck/ids
> >>> > 2014-11-05 23:39:02,866
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 5,1  replyHeader:: 5,33,0  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/ids/TestCheck_ip-172-31-25-198-1415230742830-f3dfc362,#7b2276657273696f6e223a312c22737562736372697074696f6e223a7b22486561727462656174223a317d2c227061747465726e223a22737461746963222c2274696d657374616d70223a2231343135323330373432383433227d,v{s{31,s{'world,'anyone}}},1
> >>> > response::
> >>> >
> >>>
> '/consumers/TestCheck/ids/TestCheck_ip-172-31-25-198-1415230742830-f3dfc362
> >>> > 2014-11-05 23:39:02,867 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], end registering
> >>> > consumer TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 in ZK
> >>> > 2014-11-05 23:39:02,868 pool-13-thread-2  DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Adding
> >>> topicThreadId
> >>> > (Test,TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0) and queue
> >>> [] to
> >>> > topicThreadIdAndQueues data structure
> >>> > 2014-11-05 23:39:02,868
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], starting watcher
> >>> > executor thread for consumer
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362
> >>> > 2014-11-05 23:39:02,868 pool-13-thread-2  DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], adding topic
> Test
> >>> and
> >>> > 1 streams to map.
> >>> > 2014-11-05 23:39:02,870
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 6,3  replyHeader:: 6,33,0  request::
> >>> > '/consumers/TestCheck/ids,T  response::
> >>> > s{32,32,1415230742859,1415230742859,0,1,0,0,0,1,33}
> >>> > 2014-11-05 23:39:02,872
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 7,8  replyHeader:: 7,33,0  request::
> >>> > '/consumers/TestCheck/ids,T  response::
> >>> > v{'TestCheck_ip-172-31-25-198-1415230742830-f3dfc362}
> >>> > 2014-11-05 23:39:02,873
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 8,3  replyHeader:: 8,33,-101  request::
> >>> > '/brokers/topics/Test,T  response::
> >>> > 2014-11-05 23:39:02,873 pool-13-thread-2  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-137: Subscribed data changes for
> >>> > /brokers/topics/Test
> >>> > 2014-11-05 23:39:02,873 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], begin
> rebalancing
> >>> > consumer TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 try #0
> >>> > 2014-11-05 23:39:02,875 pool-13-thread-1  DEBUG
> >>> > kafka.producer.async.DefaultEventHandler-52: Handling 1 events
> >>> > 2014-11-05 23:39:02,877
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 9,8  replyHeader:: 9,33,0  request::
> >>> > '/brokers/ids,F  response:: v{'0}
> >>> > 2014-11-05 23:39:02,879
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 10,4  replyHeader:: 10,33,0  request::
> >>> > '/brokers/ids/0,F  response::
> >>> >
> >>> >
> >>>
> #7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343135323330363632383839222c22686f7374223a2235342e3231392e3134332e313639222c2276657273696f6e223a312c22706f7274223a393039327d,s{20,20,1415230662902,1415230662902,0,0,0,92748555640897537,91,0,20}
> >>> >
> >>> > 2014-11-05 23:39:02,887
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 11,4  replyHeader:: 11,33,0  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/ids/TestCheck_ip-172-31-25-198-1415230742830-f3dfc362,F
> >>> > response::
> >>> >
> >>> >
> >>>
> #7b2276657273696f6e223a312c22737562736372697074696f6e223a7b22486561727462656174223a317d2c227061747465726e223a22737461746963222c2274696d657374616d70223a2231343135323330373432383433227d,s{33,33,1415230742861,1415230742861,0,0,0,92748555640897539,91,0,33}
> >>> >
> >>> > 2014-11-05 23:39:02,896
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 12,8  replyHeader:: 12,33,0  request::
> >>> > '/consumers/TestCheck/ids,T  response::
> >>> > v{'TestCheck_ip-172-31-25-198-1415230742830-f3dfc362}
> >>> > 2014-11-05 23:39:02,898
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 13,4  replyHeader:: 13,33,0  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/ids/TestCheck_ip-172-31-25-198-1415230742830-f3dfc362,F
> >>> > response::
> >>> >
> >>> >
> >>>
> #7b2276657273696f6e223a312c22737562736372697074696f6e223a7b22486561727462656174223a317d2c227061747465726e223a22737461746963222c2274696d657374616d70223a2231343135323330373432383433227d,s{33,33,1415230742861,1415230742861,0,0,0,92748555640897539,91,0,33}
> >>> >
> >>> > 2014-11-05 23:39:02,908
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 14,8  replyHeader:: 14,33,0  request::
> >>> > '/brokers/ids,F  response:: v{'0}
> >>> > 2014-11-05 23:39:02,910
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 15,4  replyHeader:: 15,33,0  request::
> >>> > '/brokers/ids/0,F  response::
> >>> >
> >>> >
> >>>
> #7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343135323330363632383839222c22686f7374223a2235342e3231392e3134332e313639222c2276657273696f6e223a312c22706f7274223a393039327d,s{20,20,1415230662902,1415230662902,0,0,0,92748555640897537,91,0,20}
> >>> >
> >>> > 2014-11-05 23:39:02,917 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 0 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:02,917
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 16,4  replyHeader:: 16,33,-101  request::
> >>> > '/brokers/topics/Test,T  response::
> >>> > 2014-11-05 23:39:02,920 pool-13-thread-2  DEBUG
> >>> kafka.utils.ZkUtils$-52:
> >>> > Partition map for /brokers/topics/Test is Map()
> >>> > 2014-11-05 23:39:02,920 pool-13-thread-2   INFO
> >>> > kafka.consumer.ConsumerFetcherManager-68:
> >>> > [ConsumerFetcherManager-1415230742842] Stopping leader finder thread
> >>> > 2014-11-05 23:39:02,921 pool-13-thread-2   INFO
> >>> > kafka.consumer.ConsumerFetcherManager-68:
> >>> > [ConsumerFetcherManager-1415230742842] Stopping all fetchers
> >>> > 2014-11-05 23:39:02,921 pool-13-thread-2   INFO
> >>> > kafka.consumer.ConsumerFetcherManager-68:
> >>> > [ConsumerFetcherManager-1415230742842] All connections stopped
> >>> > 2014-11-05 23:39:02,921 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Cleared all
> >>> relevant
> >>> > queues for this fetcher
> >>> > 2014-11-05 23:39:02,921 pool-13-thread-2  DEBUG
> >>> > kafka.consumer.ConsumerIterator-52: Clearing the current data chunk
> for
> >>> > this consumer iterator
> >>> > 2014-11-05 23:39:02,921 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Cleared the data
> >>> > chunks in all the consumer message iterators
> >>> > 2014-11-05 23:39:02,922 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Committing all
> >>> offsets
> >>> > after clearing the fetcher queues
> >>> > 2014-11-05 23:39:02,922 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Releasing
> >>> partition
> >>> > ownership
> >>> > 2014-11-05 23:39:02,922 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Consumer
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 rebalancing the
> >>> following
> >>> > partitions: ArrayBuffer() for topic Test with consumers:
> >>> > List(TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0)
> >>> > 2014-11-05 23:39:02,922 pool-13-thread-2   WARN
> >>> > kafka.consumer.ZookeeperConsumerConnector-83:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], No broker
> >>> partitions
> >>> > consumed by consumer thread
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0 for topic Test
> >>> > 2014-11-05 23:39:02,922 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Updating the
> cache
> >>> > 2014-11-05 23:39:02,923 pool-13-thread-2  DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Partitions per
> >>> topic
> >>> > cache Map(Test -> ArrayBuffer())
> >>> > 2014-11-05 23:39:02,923 pool-13-thread-2  DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Consumers per
> >>> topic
> >>> > cache Map(Test ->
> >>> > List(TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0))
> >>> > 2014-11-05 23:39:02,923 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Consumer
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 selected
> partitions :
> >>> > 2014-11-05 23:39:02,927 pool-13-thread-2   INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], end rebalancing
> >>> > consumer TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 try #0
> >>> > 2014-11-05 23:39:02,928
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.consumer.ConsumerFetcherManager$LeaderFinderThread-68:
> >>> >
> >>>
> [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread],
> >>> > Starting
> >>> > 2014-11-05 23:39:02,930 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:02,930 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:02,985
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-783: Got notification
> >>> > sessionid:0x1498251e8680003
> >>> > 2014-11-05 23:39:02,986
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-800: Got WatchedEvent
> >>> > state:SyncConnected type:NodeCreated path:/brokers/topics/Test for
> >>> > sessionid 0x1498251e8680003
> >>> > 2014-11-05 23:39:02,987 pool-13-thread-2-EventThread  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-351: Received event: WatchedEvent
> >>> > state:SyncConnected type:NodeCreated path:/brokers/topics/Test
> >>> > 2014-11-05 23:39:02,987 pool-13-thread-2-EventThread  DEBUG
> >>> > org.I0Itec.zkclient.ZkEventThread-88: New event: ZkEvent[Data of
> >>> > /brokers/topics/Test changed sent to
> >>> >
> >>> >
> >>>
> kafka.consumer.ZookeeperConsumerConnector$ZKTopicPartitionChangeListener@523b24ee
> >>> > ]
> >>> > 2014-11-05 23:39:02,987 pool-13-thread-2-EventThread  DEBUG
> >>> > org.I0Itec.zkclient.ZkClient-395: Leaving process event
> >>> > 2014-11-05 23:39:02,988 ZkClient-EventThread-29-172.31.25.198:2181
> >>> DEBUG
> >>> > org.I0Itec.zkclient.ZkEventThread-69: Delivering event #1
> ZkEvent[Data
> >>> of
> >>> > /brokers/topics/Test changed sent to
> >>> >
> >>> >
> >>>
> kafka.consumer.ZookeeperConsumerConnector$ZKTopicPartitionChangeListener@523b24ee
> >>> > ]
> >>> > 2014-11-05 23:39:02,991
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 17,3  replyHeader:: 17,37,0  request::
> >>> > '/brokers/topics/Test,T  response::
> >>> > s{37,37,1415230742983,1415230742983,0,0,0,0,44,0,37}
> >>> > 2014-11-05 23:39:02,993
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 18,4  replyHeader:: 18,37,0  request::
> >>> > '/brokers/topics/Test,T  response::
> >>> >
> >>> >
> >>>
> #7b2276657273696f6e223a312c22706172746974696f6e73223a7b2231223a5b305d2c2230223a5b305d7d7d,s{37,37,1415230742983,1415230742983,0,0,0,0,44,0,37}
> >>> >
> >>> > 2014-11-05 23:39:02,994 ZkClient-EventThread-29-172.31.25.198:2181
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Topic info for
> >>> path
> >>> > /brokers/topics/Test changed to
> >>> > {"version":1,"partitions":{"1":[0],"0":[0]}}, triggering rebalance
> >>> > 2014-11-05 23:39:02,994 ZkClient-EventThread-29-172.31.25.198:2181
> >>> DEBUG
> >>> > org.I0Itec.zkclient.ZkEventThread-79: Delivering event #1 done
> >>> > 2014-11-05 23:39:02,995
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], begin
> rebalancing
> >>> > consumer TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 try #0
> >>> > 2014-11-05 23:39:02,996
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 19,8  replyHeader:: 19,37,0  request::
> >>> > '/brokers/ids,F  response:: v{'0}
> >>> > 2014-11-05 23:39:02,998
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 20,4  replyHeader:: 20,37,0  request::
> >>> > '/brokers/ids/0,F  response::
> >>> >
> >>> >
> >>>
> #7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343135323330363632383839222c22686f7374223a2235342e3231392e3134332e313639222c2276657273696f6e223a312c22706f7274223a393039327d,s{20,20,1415230662902,1415230662902,0,0,0,92748555640897537,91,0,20}
> >>> >
> >>> > 2014-11-05 23:39:03,008
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 21,4  replyHeader:: 21,37,0  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/ids/TestCheck_ip-172-31-25-198-1415230742830-f3dfc362,F
> >>> > response::
> >>> >
> >>> >
> >>>
> #7b2276657273696f6e223a312c22737562736372697074696f6e223a7b22486561727462656174223a317d2c227061747465726e223a22737461746963222c2274696d657374616d70223a2231343135323330373432383433227d,s{33,33,1415230742861,1415230742861,0,0,0,92748555640897539,91,0,33}
> >>> >
> >>> > 2014-11-05 23:39:03,014
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 22,8  replyHeader:: 22,37,0  request::
> >>> > '/consumers/TestCheck/ids,T  response::
> >>> > v{'TestCheck_ip-172-31-25-198-1415230742830-f3dfc362}
> >>> > 2014-11-05 23:39:03,018
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 23,4  replyHeader:: 23,37,0  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/ids/TestCheck_ip-172-31-25-198-1415230742830-f3dfc362,F
> >>> > response::
> >>> >
> >>> >
> >>>
> #7b2276657273696f6e223a312c22737562736372697074696f6e223a7b22486561727462656174223a317d2c227061747465726e223a22737461746963222c2274696d657374616d70223a2231343135323330373432383433227d,s{33,33,1415230742861,1415230742861,0,0,0,92748555640897539,91,0,33}
> >>> >
> >>> > 2014-11-05 23:39:03,041
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 24,8  replyHeader:: 24,37,0  request::
> >>> > '/brokers/ids,F  response:: v{'0}
> >>> > 2014-11-05 23:39:03,043
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 25,4  replyHeader:: 25,37,0  request::
> >>> > '/brokers/ids/0,F  response::
> >>> >
> >>> >
> >>>
> #7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343135323330363632383839222c22686f7374223a2235342e3231392e3134332e313639222c2276657273696f6e223a312c22706f7274223a393039327d,s{20,20,1415230662902,1415230662902,0,0,0,92748555640897537,91,0,20}
> >>> >
> >>> > 2014-11-05 23:39:03,058
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 26,4  replyHeader:: 26,37,0  request::
> >>> > '/brokers/topics/Test,T  response::
> >>> >
> >>> >
> >>>
> #7b2276657273696f6e223a312c22706172746974696f6e73223a7b2231223a5b305d2c2230223a5b305d7d7d,s{37,37,1415230742983,1415230742983,0,0,0,0,44,0,37}
> >>> >
> >>> > 2014-11-05 23:39:03,074
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.utils.ZkUtils$-52: Partition map for /brokers/topics/Test is
> >>> Map(1 ->
> >>> > List(0), 0 -> List(0))
> >>> > 2014-11-05 23:39:03,075
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ConsumerFetcherManager-68:
> >>> > [ConsumerFetcherManager-1415230742842] Stopping leader finder thread
> >>> > 2014-11-05 23:39:03,082
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread-68:
> >>> >
> >>>
> [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread],
> >>> > Shutting down
> >>> > 2014-11-05 23:39:03,082 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:03,089
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.consumer.ConsumerFetcherManager$LeaderFinderThread-68:
> >>> >
> >>>
> [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread],
> >>> > Stopped
> >>> > 2014-11-05 23:39:03,089
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread-68:
> >>> >
> >>>
> [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread],
> >>> > Shutdown completed
> >>> > 2014-11-05 23:39:03,089
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ConsumerFetcherManager-68:
> >>> > [ConsumerFetcherManager-1415230742842] Stopping all fetchers
> >>> > 2014-11-05 23:39:03,090
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ConsumerFetcherManager-68:
> >>> > [ConsumerFetcherManager-1415230742842] All connections stopped
> >>> > 2014-11-05 23:39:03,090
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Cleared all
> >>> relevant
> >>> > queues for this fetcher
> >>> > 2014-11-05 23:39:03,090
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.ConsumerIterator-52: Clearing the current data chunk
> for
> >>> > this consumer iterator
> >>> > 2014-11-05 23:39:03,091
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Cleared the data
> >>> > chunks in all the consumer message iterators
> >>> > 2014-11-05 23:39:03,091
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Committing all
> >>> offsets
> >>> > after clearing the fetcher queues
> >>> > 2014-11-05 23:39:03,091
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Releasing
> >>> partition
> >>> > ownership
> >>> > 2014-11-05 23:39:03,094 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:03,096
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Consumer
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 rebalancing the
> >>> following
> >>> > partitions: ArrayBuffer(0, 1) for topic Test with consumers:
> >>> > List(TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0)
> >>> > 2014-11-05 23:39:03,098
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362],
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0 attempting to
> claim
> >>> > partition 0
> >>> > 2014-11-05 23:39:03,100
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 27,4  replyHeader:: 27,37,-101  request::
> >>> > '/consumers/TestCheck/offsets/Test/0,F  response::
> >>> > 2014-11-05 23:39:03,116
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.PartitionTopicInfo-52: initial consumer offset of
> >>> Test:0:
> >>> > fetched offset = -1: consumed offset = -1 is -1
> >>> > 2014-11-05 23:39:03,117
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.PartitionTopicInfo-52: initial fetch offset of Test:0:
> >>> > fetched offset = -1: consumed offset = -1 is -1
> >>> > 2014-11-05 23:39:03,118
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Test:0: fetched
> >>> offset
> >>> > = -1: consumed offset = -1 selected new offset -1
> >>> > 2014-11-05 23:39:03,169
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362],
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0 attempting to
> claim
> >>> > partition 1
> >>> > 2014-11-05 23:39:03,171
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 28,4  replyHeader:: 28,37,-101  request::
> >>> > '/consumers/TestCheck/offsets/Test/1,F  response::
> >>> > 2014-11-05 23:39:03,172
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.PartitionTopicInfo-52: initial consumer offset of
> >>> Test:1:
> >>> > fetched offset = -1: consumed offset = -1 is -1
> >>> > 2014-11-05 23:39:03,172
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.PartitionTopicInfo-52: initial fetch offset of Test:1:
> >>> > fetched offset = -1: consumed offset = -1 is -1
> >>> > 2014-11-05 23:39:03,172
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Test:1: fetched
> >>> offset
> >>> > = -1: consumed offset = -1 selected new offset -1
> >>> > 2014-11-05 23:39:03,175 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:03,175
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 29,1  replyHeader:: 29,38,-101  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/owners/Test/1,#6865616c7468636865636b5f69702d3137322d33312d32352d3139382d313431353233303734323833302d66336466633336322d30,v{s{31,s{'world,'anyone}}},1
> >>> > response::
> >>> > 2014-11-05 23:39:03,179
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 30,1  replyHeader:: 30,39,-101  request::
> >>> > '/consumers/TestCheck/owners/Test,,v{s{31,s{'world,'anyone}}},0
> >>> > response::
> >>> > 2014-11-05 23:39:03,180 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:03,181 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 1 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:03,182 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:03,182 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:03,186
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 31,1  replyHeader:: 31,40,0  request::
> >>> > '/consumers/TestCheck/owners,,v{s{31,s{'world,'anyone}}},0
> response::
> >>> > '/consumers/TestCheck/owners
> >>> > 2014-11-05 23:39:03,189 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:03,190
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 32,1  replyHeader:: 32,41,0  request::
> >>> > '/consumers/TestCheck/owners/Test,,v{s{31,s{'world,'anyone}}},0
> >>> response::
> >>> > '/consumers/TestCheck/owners/Test
> >>> > 2014-11-05 23:39:03,190 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:03,191 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:03,192 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:03,193
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 33,1  replyHeader:: 33,42,0  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/owners/Test/1,#6865616c7468636865636b5f69702d3137322d33312d32352d3139382d313431353233303734323833302d66336466633336322d30,v{s{31,s{'world,'anyone}}},1
> >>> > response:: '/consumers/TestCheck/owners/Test/1
> >>> > 2014-11-05 23:39:03,193 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 10
> >>> > 2014-11-05 23:39:03,193
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362],
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0 successfully
> owned
> >>> > partition 1 for topic Test
> >>> > 2014-11-05 23:39:03,196
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 34,1  replyHeader:: 34,43,0  request::
> >>> >
> >>> >
> >>>
> '/consumers/TestCheck/owners/Test/0,#6865616c7468636865636b5f69702d3137322d33312d32352d3139382d313431353233303734323833302d66336466633336322d30,v{s{31,s{'world,'anyone}}},1
> >>> > response:: '/consumers/TestCheck/owners/Test/0
> >>> > 2014-11-05 23:39:03,196
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362],
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0 successfully
> owned
> >>> > partition 0 for topic Test
> >>> > 2014-11-05 23:39:03,196
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Updating the
> cache
> >>> > 2014-11-05 23:39:03,197
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Partitions per
> >>> topic
> >>> > cache Map(Test -> ArrayBuffer(0, 1))
> >>> > 2014-11-05 23:39:03,197
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>> DEBUG
> >>> > kafka.consumer.ZookeeperConsumerConnector-52:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Consumers per
> >>> topic
> >>> > cache Map(Test ->
> >>> > List(TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-0))
> >>> > 2014-11-05 23:39:03,197
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], Consumer
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 selected
> partitions :
> >>> > Test:0: fetched offset = -1: consumed offset = -1,Test:1: fetched
> >>> offset =
> >>> > -1: consumed offset = -1
> >>> > 2014-11-05 23:39:03,199
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.consumer.ConsumerFetcherManager$LeaderFinderThread-68:
> >>> >
> >>>
> [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread],
> >>> > Starting
> >>> > 2014-11-05 23:39:03,199
> >>> > TestCheck_ip-172-31-25-198-1415230742830-f3dfc362_watcher_executor
> >>>  INFO
> >>> > kafka.consumer.ZookeeperConsumerConnector-68:
> >>> > [TestCheck_ip-172-31-25-198-1415230742830-f3dfc362], end rebalancing
> >>> > consumer TestCheck_ip-172-31-25-198-1415230742830-f3dfc362 try #0
> >>> > 2014-11-05 23:39:03,202
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 35,8  replyHeader:: 35,43,0  request::
> >>> > '/brokers/ids,F  response:: v{'0}
> >>> > 2014-11-05 23:39:03,204
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-839: Reading reply
> >>> > sessionid:0x1498251e8680003, packet:: clientPath:null serverPath:null
> >>> > finished:false header:: 36,4  replyHeader:: 36,43,0  request::
> >>> > '/brokers/ids/0,F  response::
> >>> >
> >>> >
> >>>
> #7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343135323330363632383839222c22686f7374223a2235342e3231392e3134332e313639222c2276657273696f6e223a312c22706f7274223a393039327d,s{20,20,1415230662902,1415230662902,0,0,0,92748555640897537,91,0,20}
> >>> >
> >>> > 2014-11-05 23:39:03,214
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.utils.VerifiableProperties-68: Verifying properties
> >>> > 2014-11-05 23:39:03,215
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.utils.VerifiableProperties-68: Property client.id is
> >>> overridden
> >>> > to TestCheck
> >>> > 2014-11-05 23:39:03,215
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.utils.VerifiableProperties-68: Property
> >>> metadata.broker.list is
> >>> > overridden to x.x.x.x:9092
> >>> > 2014-11-05 23:39:03,215
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.utils.VerifiableProperties-68: Property
> request.timeout.ms
> >>> is
> >>> > overridden to 30000
> >>> > 2014-11-05 23:39:03,217
> >>> >
> TestCheck_ip-172-31-25-198-1415230742830-f3dfc362-leader-finder-thread
> >>> > INFO kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:x.x.x.x,port:9092 with correlation id 0 for 1 topic(s)
> >>> Set(Test)
> >>> > 2014-11-05 23:39:04,195 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 2 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:04,196 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:04,196 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:04,199 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:04,199 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:04,200 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:04,200 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:04,201 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 3 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:04,201 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:04,201 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:04,205 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:04,205 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:04,206 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:04,206 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:04,206 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 9
> >>> > 2014-11-05 23:39:04,698
> >>> > Thread-3-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680002 after 0ms
> >>> > 2014-11-05 23:39:05,205
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680003 after 0ms
> >>> > 2014-11-05 23:39:05,207 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 4 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:05,208 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:05,208 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:05,211 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:05,211 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:05,212 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:05,212 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:05,212 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 5 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:05,213 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:05,213 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:05,217 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:05,217 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:05,217 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:05,217 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:05,218 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 8
> >>> > 2014-11-05 23:39:06,218 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 6 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:06,219 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:06,219 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:06,224 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:06,224 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:06,225 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:06,225 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:06,226 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 7 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:06,226 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:06,226 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:06,230 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:06,230 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:06,231 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:06,231 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:06,231 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 7
> >>> > 2014-11-05 23:39:06,699
> >>> > Thread-3-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680002 after 0ms
> >>> > 2014-11-05 23:39:07,208
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680003 after 0ms
> >>> > 2014-11-05 23:39:07,232 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 8 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:07,233 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:07,233 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:07,236 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:07,236 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:07,237 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:07,237 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:07,238 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 9 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:07,238 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:07,238 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:07,241 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:07,241 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:07,242 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:07,242 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:07,242 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 6
> >>> > 2014-11-05 23:39:08,243 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 10 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:08,243 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:08,245 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:08,249 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:08,250 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:08,250 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:08,251 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:08,251 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 11 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:08,251 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:08,252 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:08,255 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:08,256 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:08,256 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:08,257 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:08,257 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 5
> >>> > 2014-11-05 23:39:08,702
> >>> > Thread-3-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680002 after 0ms
> >>> > 2014-11-05 23:39:09,211
> >>> >
> >>> >
> >>>
> pool-13-thread-2-SendThread(ip-172-31-25-198.us-west-1.compute.internal:2181)
> >>> > DEBUG org.apache.zookeeper.ClientCnxn-759: Got ping response for
> >>> sessionid:
> >>> > 0x1498251e8680003 after 0ms
> >>> > 2014-11-05 23:39:09,257 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 12 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:09,258 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:09,258 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:09,261 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:09,262 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:09,262 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:09,262 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:09,263 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 13 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:09,264 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:09,264 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:09,268 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:09,268 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:09,269 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:09,269 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:09,269 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 4
> >>> > 2014-11-05 23:39:10,270 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 14 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:10,271 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:10,271 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:10,275 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:10,275 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:10,275 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:10,276 pool-13-thread-1  DEBUG
> >>> > kafka.producer.BrokerPartitionInfo-52: Getting broker partition info
> >>> for
> >>> > topic Test
> >>> > 2014-11-05 23:39:10,276 pool-13-thread-1   INFO
> >>> > kafka.client.ClientUtils$-68: Fetching metadata from broker
> >>> > id:0,host:172.31.25.198,port:9092 with correlation id 15 for 1
> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:10,277 pool-13-thread-1  DEBUG
> >>> > kafka.network.BlockingChannel-52: Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > (requested 10000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > (requested 102400).
> >>> > 2014-11-05 23:39:10,277 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Connected to 172.31.25.198:9092 for
> >>> > producing
> >>> > 2014-11-05 23:39:10,280 pool-13-thread-1   INFO
> >>> > kafka.producer.SyncProducer-68: Disconnecting from
> 172.31.25.198:9092
> >>> > 2014-11-05 23:39:10,280 pool-13-thread-1  DEBUG
> >>> > kafka.client.ClientUtils$-52: Successfully fetched metadata for 1
> >>> topic(s)
> >>> > Set(Test)
> >>> > 2014-11-05 23:39:10,280 pool-13-thread-1   WARN
> >>> > kafka.producer.BrokerPartitionInfo-83: Error while fetching metadata
> >>> > [{TopicMetadata for topic Test ->
> >>> > No partition metadata for topic Test due to
> >>> > kafka.common.LeaderNotAvailableException}] for topic [Test]: class
> >>> > kafka.common.LeaderNotAvailableException
> >>> > 2014-11-05 23:39:10,280 pool-13-thread-1  ERROR
> >>> > kafka.producer.async.DefaultEventHandler-97: Failed to collate
> >>> messages by
> >>> > topic, partition due to: Failed to fetch topic metadata for topic:
> Test
> >>> > 2014-11-05 23:39:10,280 pool-13-thread-1   INFO
> >>> > kafka.producer.async.DefaultEventHandler-68: Back off for 1000 ms
> >>> before
> >>> > retrying send. Remaining retries = 3
> >>> >
> >>> >
> >>> > Brokers Logs -
> >>> >
> >>> > [2014-11-05 23:39:02,983] INFO Topic creation
> >>> > {"version":1,"partitions":{"1":[0],"0":[0]}}
> (kafka.admin.AdminUtils$)
> >>> > [2014-11-05 23:39:02,988] INFO [KafkaApi-0] Auto creation of topic
> Test
> >>> > with 2 partitions and replication factor 1 is successful!
> >>> > (kafka.server.KafkaApis)
> >>> > [2014-11-05 23:39:03,092] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:03,191] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:04,200] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:04,205] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:05,211] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:05,218] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:06,224] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:06,230] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:07,236] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> > [2014-11-05 23:39:07,241] INFO Closing socket connection to /
> >>> 172.31.25.198
> >>> > .
> >>> > (kafka.network.Processor)
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > On Wed, Nov 5, 2014 at 3:05 PM, Guozhang Wang <wangg...@gmail.com>
> >>> wrote:
> >>> >
> >>> > > Sameer,
> >>> > >
> >>> > > Yes, this is the server log. But there seems no abnormal entries in
> >>> it,
> >>> > and
> >>> > > it does not cover the same time range as the producer client
> throwing
> >>> > > LeaderNotAvailableException
> >>> > > (it was 10/24, 14:30).
> >>> > >
> >>> > > The reason that I want to check the server log at that same reason
> is
> >>> > > that LeaderNotAvailableException
> >>> > > is set by the brokers while handling requests, and its server log
> >>> will
> >>> > show
> >>> > > what is the cause it set this error code.
> >>> > >
> >>> > > Guozhang
> >>> > >
> >>> > > On Tue, Nov 4, 2014 at 12:47 PM, Sameer Yami <sy...@ailive.net>
> >>> wrote:
> >>> > >
> >>> > > > Hi Guozhang,
> >>> > > >
> >>> > > > This is the server.log -
> >>> > > >
> >>> > > > [2014-11-04 20:21:57,510] INFO Verifying properties
> >>> > > > (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,545] INFO Property advertised.host.name is
> >>> > > overridden
> >>> > > > to x.x.x.x (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,545] INFO Property broker.id is overridden
> >>> to 0
> >>> > > > (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,545] INFO Property
> controlled.shutdown.enable
> >>> is
> >>> > > > overridden to true (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,545] INFO Property host.name is overridden
> to
> >>> > > > 172.31.25.198 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,545] INFO Property log.cleaner.enable is
> >>> > overridden
> >>> > > to
> >>> > > > false (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,546] INFO Property log.dirs is overridden to
> >>> > > > /tmp/kafka-logs (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,546] INFO Property
> >>> > log.retention.check.interval.ms
> >>> > > is
> >>> > > > overridden to 60000 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,546] INFO Property log.retention.hours is
> >>> > overridden
> >>> > > > to 168 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,546] INFO Property log.segment.bytes is
> >>> overridden
> >>> > > to
> >>> > > > 536870912 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,546] INFO Property num.io.threads is
> >>> overridden
> >>> > to 8
> >>> > > > (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,547] INFO Property num.network.threads is
> >>> > overridden
> >>> > > > to 2 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,547] INFO Property num.partitions is
> >>> overridden
> >>> > to 2
> >>> > > > (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,547] INFO Property port is overridden to
> 9092
> >>> > > > (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,547] INFO Property
> >>> socket.receive.buffer.bytes is
> >>> > > > overridden to 1048576 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,547] INFO Property socket.request.max.bytes
> is
> >>> > > > overridden to 104857600 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,547] INFO Property socket.send.buffer.bytes
> is
> >>> > > > overridden to 1048576 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,548] INFO Property zookeeper.connect is
> >>> overridden
> >>> > > to
> >>> > > > localhost:2181 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,548] INFO Property
> >>> > zookeeper.connection.timeout.ms
> >>> > > is
> >>> > > > overridden to 1000000 (kafka.utils.VerifiableProperties)
> >>> > > > [2014-11-04 20:21:57,563] INFO [Kafka Server 0], starting
> >>> > > > (kafka.server.KafkaServer)
> >>> > > > [2014-11-04 20:21:57,565] INFO [Kafka Server 0], Connecting to
> >>> > zookeeper
> >>> > > on
> >>> > > > localhost:2181 (kafka.server.KafkaServer)
> >>> > > > [2014-11-04 20:21:57,738] INFO Log directory '/tmp/kafka-logs'
> not
> >>> > found,
> >>> > > > creating it. (kafka.log.LogManager)
> >>> > > > [2014-11-04 20:21:57,748] INFO Starting log cleanup with a period
> >>> of
> >>> > > 60000
> >>> > > > ms. (kafka.log.LogManager)
> >>> > > > [2014-11-04 20:21:57,752] INFO Starting log flusher with a
> default
> >>> > period
> >>> > > > of 9223372036854775807 ms. (kafka.log.LogManager)
> >>> > > > [2014-11-04 20:21:57,783] INFO Awaiting socket connections on
> >>> > > > ip-172-31-25-198.us-west-1.compute.internal:9092.
> >>> > > (kafka.network.Acceptor)
> >>> > > > [2014-11-04 20:21:57,784] INFO [Socket Server on Broker 0],
> Started
> >>> > > > (kafka.network.SocketServer)
> >>> > > > [2014-11-04 20:21:57,852] INFO Will not load MX4J, mx4j-tools.jar
> >>> is
> >>> > not
> >>> > > in
> >>> > > > the classpath (kafka.utils.Mx4jLoader$)
> >>> > > > [2014-11-04 20:21:57,884] INFO 0 successfully elected as leader
> >>> > > > (kafka.server.ZookeeperLeaderElector)
> >>> > > > [2014-11-04 20:21:57,967] INFO Registered broker 0 at path
> >>> > /brokers/ids/0
> >>> > > > with address x.x.x.x:9092. (kafka.utils.ZkUtils$)
> >>> > > > [2014-11-04 20:21:57,981] INFO [Kafka Server 0], started
> >>> > > > (kafka.server.KafkaServer)
> >>> > > > [2014-11-04 20:21:58,050] INFO New leader is 0
> >>> > > > (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> >>> > > > [2014-11-04 20:22:01,809] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:22:11,812] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:29,084] INFO Topic creation
> >>> > > > {"version":1,"partitions":{"1":[0],"0":[0]}}
> >>> (kafka.admin.AdminUtils$)
> >>> > > > [2014-11-04 20:30:29,088] INFO [KafkaApi-0] Auto creation of
> topic
> >>> Test
> >>> > > > with 2 partitions and replication factor 1 is successful!
> >>> > > > (kafka.server.KafkaApis)
> >>> > > > [2014-11-04 20:30:29,280] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:29,301] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:30,310] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:30,315] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:31,320] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:31,325] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:32,331] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:32,336] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:33,343] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:33,347] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:34,356] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:34,360] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:35,366] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:35,372] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:36,377] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:36,382] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:37,387] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:37,394] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:38,399] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:38,403] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:39,410] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:39,414] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:40,419] INFO Closing socket connection to /
> >>> > > 172.31.25.198
> >>> > > > .
> >>> > > > (kafka.network.Processor)
> >>> > > > [2014-11-04 20:30:41,602] INFO Topic creation
> >>> > > > {"version":1,"partitions":{"1":[0],"0":[0]}}
> >>> (kafka.admin.AdminUtils$)
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > > thanks
> >>> > > >
> >>> > > >
> >>> > > > On Fri, Oct 31, 2014 at 9:15 AM, Guozhang Wang <
> wangg...@gmail.com
> >>> >
> >>> > > wrote:
> >>> > > >
> >>> > > > > The server.log should be the broker logs.
> >>> > > > >
> >>> > > > > Guozhang
> >>> > > > >
> >>> > > > > On Thu, Oct 30, 2014 at 11:32 AM, Sameer Yami <
> sy...@ailive.net>
> >>> > > wrote:
> >>> > > > >
> >>> > > > > > I do not see any other logs besides these.
> >>> > > > > > I checked the log4j.properties and where the logs are getting
> >>> > stored.
> >>> > > > > > The only other logs (apart from the controller logs) in the
> >>> > directory
> >>> > > > are
> >>> > > > > > zookeeper-gc.log,kafkaServer-gc.log, server.log and
> >>> > state-change.log
> >>> > > > > >
> >>> > > > > > Thanks
> >>> > > > > >
> >>> > > > > > On Wed, Oct 29, 2014 at 4:13 PM, Guozhang Wang <
> >>> wangg...@gmail.com
> >>> > >
> >>> > > > > wrote:
> >>> > > > > >
> >>> > > > > > > Sorry for getting late on this Sameer.
> >>> > > > > > >
> >>> > > > > > > The logs you showed above are from controller logs, do you
> >>> see
> >>> > > > brokers
> >>> > > > > > logs
> >>> > > > > > > besides that?
> >>> > > > > > >
> >>> > > > > > > Guozhang
> >>> > > > > > >
> >>> > > > > > > On Wed, Oct 29, 2014 at 3:40 PM, Sameer Yami <
> >>> sy...@ailive.net>
> >>> > > > wrote:
> >>> > > > > > >
> >>> > > > > > > > Hi Guozhang,
> >>> > > > > > > >
> >>> > > > > > > > Is there any clue in the above data as to what is wrong?
> >>> > > > > > > >
> >>> > > > > > > > thanks
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > On Tue, Oct 28, 2014 at 3:45 PM, Sameer Yami <
> >>> sy...@ailive.net
> >>> > >
> >>> > > > > wrote:
> >>> > > > > > > >
> >>> > > > > > > > > This is the one from the file specified in
> >>> log4j.properties -
> >>> > > > > > > > >
> >>> > > > > > > > > [2014-10-28 22:38:36,082] DEBUG [TopicChangeListener on
> >>> > > > Controller
> >>> > > > > > 0]:
> >>> > > > > > > > > Topic change listener fired for path /brokers/topics
> with
> >>> > > > children
> >>> > > > > > Test
> >>> > > > > > > > >
> >>> (kafka.controller.PartitionStateMachine$TopicChangeListener)
> >>> > > > > > > > > [2014-10-28 22:38:36,094] INFO [TopicChangeListener on
> >>> > > Controller
> >>> > > > > 0]:
> >>> > > > > > > New
> >>> > > > > > > > > topics: [Set(Test)], deleted topics: [Set()], new
> >>> partition
> >>> > > > replica
> >>> > > > > > > > > assignment [Map([Test,0] -> List(0), [Test,1] ->
> >>> List(0))]
> >>> > > > > > > > >
> >>> (kafka.controller.PartitionStateMachine$TopicChangeListener)
> >>> > > > > > > > > [2014-10-28 22:38:36,094] INFO [Controller 0]: New
> topic
> >>> > > creation
> >>> > > > > > > > callback
> >>> > > > > > > > > for [Test,0],[Test,1]
> (kafka.controller.KafkaController)
> >>> > > > > > > > > [2014-10-28 22:38:36,095] INFO [Controller 0]: New
> >>> partition
> >>> > > > > creation
> >>> > > > > > > > > callback for [Test,0],[Test,1]
> >>> > > (kafka.controller.KafkaController)
> >>> > > > > > > > > [2014-10-28 22:38:36,095] INFO [Partition state machine
> >>> on
> >>> > > > > Controller
> >>> > > > > > > 0]:
> >>> > > > > > > > > Invoking state change to NewPartition for partitions
> >>> > > > > > [Test,0],[Test,1]
> >>> > > > > > > > > (kafka.controller.PartitionStateMachine)
> >>> > > > > > > > > [2014-10-28 22:38:36,109] INFO [Replica state machine
> on
> >>> > > > controller
> >>> > > > > > 0]:
> >>> > > > > > > > > Invoking state change to NewReplica for replicas
> >>> > > > > > > > >
> >>> > > > >
> >>> [Topic=Test,Partition=0,Replica=0],[Topic=Test,Partition=1,Replica=0]
> >>> > > > > > > > > (kafka.controller.ReplicaStateMachine)
> >>> > > > > > > > > [2014-10-28 22:38:36,111] INFO [Partition state machine
> >>> on
> >>> > > > > Controller
> >>> > > > > > > 0]:
> >>> > > > > > > > > Invoking state change to OnlinePartition for partitions
> >>> > > > > > > [Test,0],[Test,1]
> >>> > > > > > > > > (kafka.controller.PartitionStateMachine)
> >>> > > > > > > > > [2014-10-28 22:38:36,111] DEBUG [Partition state
> machine
> >>> on
> >>> > > > > > Controller
> >>> > > > > > > > 0]:
> >>> > > > > > > > > Live assigned replicas for partition [Test,0] are:
> >>> [List(0)]
> >>> > > > > > > > > (kafka.controller.PartitionStateMachine)
> >>> > > > > > > > > [2014-10-28 22:38:36,112] DEBUG [Partition state
> machine
> >>> on
> >>> > > > > > Controller
> >>> > > > > > > > 0]:
> >>> > > > > > > > > Initializing leader and isr for partition [Test,0] to
> >>> > > > > > > > > (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:1)
> >>> > > > > > > > > (kafka.controller.PartitionStateMachine)
> >>> > > > > > > > > [2014-10-28 22:38:36,141] DEBUG [Partition state
> machine
> >>> on
> >>> > > > > > Controller
> >>> > > > > > > > 0]:
> >>> > > > > > > > > Live assigned replicas for partition [Test,1] are:
> >>> [List(0)]
> >>> > > > > > > > > (kafka.controller.PartitionStateMachine)
> >>> > > > > > > > > [2014-10-28 22:38:36,142] DEBUG [Partition state
> machine
> >>> on
> >>> > > > > > Controller
> >>> > > > > > > > 0]:
> >>> > > > > > > > > Initializing leader and isr for partition [Test,1] to
> >>> > > > > > > > > (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:1)
> >>> > > > > > > > > (kafka.controller.PartitionStateMachine)
> >>> > > > > > > > > [2014-10-28 22:38:36,150] INFO [Replica state machine
> on
> >>> > > > controller
> >>> > > > > > 0]:
> >>> > > > > > > > > Invoking state change to OnlineReplica for replicas
> >>> > > > > > > > >
> >>> > > > >
> >>> [Topic=Test,Partition=0,Replica=0],[Topic=Test,Partition=1,Replica=0]
> >>> > > > > > > > > (kafka.controller.ReplicaStateMachine)
> >>> > > > > > > > > [2014-10-28 22:39:28,160] ERROR
> >>> > > > > > [Controller-0-to-broker-0-send-thread],
> >>> > > > > > > > > Controller 0's connection to broker
> >>> > id:0,host:x.x.x.x,port:9092
> >>> > > > was
> >>> > > > > > > > > unsuccessful (kafka.controller.RequestSendThread)
> >>> > > > > > > > > java.net.ConnectException: Connection timed out
> >>> > > > > > > > >     at sun.nio.ch.Net.connect0(Native Method)
> >>> > > > > > > > >     at sun.nio.ch.Net.connect(Net.java:457)
> >>> > > > > > > > >     at sun.nio.ch.Net.connect(Net.java:449)
> >>> > > > > > > > >     at
> >>> > > > > >
> >>> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647)
> >>> > > > > > > > >     at
> >>> > > > > >
> kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
> >>> > > > > > > > >     at
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.controller.RequestSendThread.connectToBroker(ControllerChannelManager.scala:173)
> >>> > > > > > > > >     at
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:140)
> >>> > > > > > > > >     at
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
> >>> > > > > > > > >     at
> >>> > > > > >
> kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
> >>> > > > > > > > > [2014-10-28 22:39:28,460] ERROR
> >>> > > > > > [Controller-0-to-broker-0-send-thread],
> >>> > > > > > > > > Controller 0 epoch 1 failed to send LeaderAndIsr
> request
> >>> with
> >>> > > > > > > correlation
> >>> > > > > > > > > id 7 to broker id:0,host:x.x.x.x,port:9092.
> Reconnecting
> >>> to
> >>> > > > broker.
> >>> > > > > > > > > (kafka.controller.RequestSendThread)
> >>> > > > > > > > > java.nio.channels.ClosedChannelException
> >>> > > > > > > > >     at
> >>> > > > kafka.network.BlockingChannel.send(BlockingChannel.scala:89)
> >>> > > > > > > > >     at
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
> >>> > > > > > > > >     at
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
> >>> > > > > > > > >     at
> >>> > > > > >
> kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
> >>> > > > > > > > >
> >>> > > > > > > > > On Tue, Oct 28, 2014 at 3:09 PM, Guozhang Wang <
> >>> > > > wangg...@gmail.com
> >>> > > > > >
> >>> > > > > > > > wrote:
> >>> > > > > > > > >
> >>> > > > > > > > >> Sameer, the one you sent me is from the producer logs,
> >>> not
> >>> > the
> >>> > > > > > broker
> >>> > > > > > > > >> logs.
> >>> > > > > > > > >> And /tmp/kafka-logs actually store the message log
> >>> (i.e. the
> >>> > > > > "data")
> >>> > > > > > > > >> instead of the server logs. Their locations should be
> >>> > > specified
> >>> > > > in
> >>> > > > > > > your
> >>> > > > > > > > >> log4j.properties file.
> >>> > > > > > > > >>
> >>> > > > > > > > >> On Tue, Oct 28, 2014 at 2:51 PM, Sameer Yami <
> >>> > > sy...@ailive.net>
> >>> > > > > > > wrote:
> >>> > > > > > > > >>
> >>> > > > > > > > >> > Is this what you are looking for -
> >>> > > > > > > > >> >
> >>> > > > > > > > >> > 2014-10-28 21:46:24,146 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Connected to
> >>> > > > 172.31.16.31:9092
> >>> > > > > > for
> >>> > > > > > > > >> > producing
> >>> > > > > > > > >> > 2014-10-28 21:46:24,149 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Disconnecting from
> >>> > > > > > > 172.31.16.31:9092
> >>> > > > > > > > >> > 2014-10-28 21:46:24,149 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.client.ClientUtils$-52: Successfully fetched
> >>> > metadata
> >>> > > > for
> >>> > > > > 1
> >>> > > > > > > > >> topic(s)
> >>> > > > > > > > >> > Set(Test)
> >>> > > > > > > > >> > 2014-10-28 21:46:24,150 pool-13-thread-1   WARN
> >>> > > > > > > > >> > kafka.producer.BrokerPartitionInfo-83: Error while
> >>> > fetching
> >>> > > > > > metadata
> >>> > > > > > > > >> > [{TopicMetadata for topic Test ->
> >>> > > > > > > > >> > No partition metadata for topic Test due to
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException}] for topic
> >>> > [Test]:
> >>> > > > > class
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException
> >>> > > > > > > > >> > 2014-10-28 21:46:24,150 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.producer.BrokerPartitionInfo-52: Getting
> broker
> >>> > > > partition
> >>> > > > > > info
> >>> > > > > > > > for
> >>> > > > > > > > >> > topic Test
> >>> > > > > > > > >> > 2014-10-28 21:46:24,151 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.client.ClientUtils$-68: Fetching metadata from
> >>> > broker
> >>> > > > > > > > >> > id:0,host:172.31.16.31,port:9092 with correlation
> id 9
> >>> > for 1
> >>> > > > > > > topic(s)
> >>> > > > > > > > >> > Set(Test)
> >>> > > > > > > > >> > 2014-10-28 21:46:24,151 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.network.BlockingChannel-52: Created socket
> with
> >>> > > > > SO_TIMEOUT =
> >>> > > > > > > > 10000
> >>> > > > > > > > >> > (requested 10000), SO_RCVBUF = 530904 (requested
> -1),
> >>> > > > SO_SNDBUF
> >>> > > > > =
> >>> > > > > > > > 102400
> >>> > > > > > > > >> > (requested 102400).
> >>> > > > > > > > >> > 2014-10-28 21:46:24,152 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Connected to
> >>> > > > 172.31.16.31:9092
> >>> > > > > > for
> >>> > > > > > > > >> > producing
> >>> > > > > > > > >> > 2014-10-28 21:46:24,154 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Disconnecting from
> >>> > > > > > > 172.31.16.31:9092
> >>> > > > > > > > >> > 2014-10-28 21:46:24,155 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.client.ClientUtils$-52: Successfully fetched
> >>> > metadata
> >>> > > > for
> >>> > > > > 1
> >>> > > > > > > > >> topic(s)
> >>> > > > > > > > >> > Set(Test)
> >>> > > > > > > > >> > 2014-10-28 21:46:24,155 pool-13-thread-1   WARN
> >>> > > > > > > > >> > kafka.producer.BrokerPartitionInfo-83: Error while
> >>> > fetching
> >>> > > > > > metadata
> >>> > > > > > > > >> > [{TopicMetadata for topic Test ->
> >>> > > > > > > > >> > No partition metadata for topic Test due to
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException}] for topic
> >>> > [Test]:
> >>> > > > > class
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException
> >>> > > > > > > > >> > 2014-10-28 21:46:24,155 pool-13-thread-1  ERROR
> >>> > > > > > > > >> > kafka.producer.async.DefaultEventHandler-97: Failed
> to
> >>> > > collate
> >>> > > > > > > > messages
> >>> > > > > > > > >> by
> >>> > > > > > > > >> > topic, partition due to: Failed to fetch topic
> >>> metadata
> >>> > for
> >>> > > > > topic:
> >>> > > > > > > > Test
> >>> > > > > > > > >> > 2014-10-28 21:46:24,156 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.async.DefaultEventHandler-68: Back
> off
> >>> for
> >>> > > 1000
> >>> > > > > ms
> >>> > > > > > > > before
> >>> > > > > > > > >> > retrying send. Remaining retries = 6
> >>> > > > > > > > >> > 2014-10-28 21:46:25,156 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.client.ClientUtils$-68: Fetching metadata from
> >>> > broker
> >>> > > > > > > > >> > id:0,host:172.31.16.31,port:9092 with correlation id
> >>> 10
> >>> > for
> >>> > > 1
> >>> > > > > > > topic(s)
> >>> > > > > > > > >> > Set(Test)
> >>> > > > > > > > >> > 2014-10-28 21:46:25,157 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.network.BlockingChannel-52: Created socket
> with
> >>> > > > > SO_TIMEOUT =
> >>> > > > > > > > 10000
> >>> > > > > > > > >> > (requested 10000), SO_RCVBUF = 530904 (requested
> -1),
> >>> > > > SO_SNDBUF
> >>> > > > > =
> >>> > > > > > > > 102400
> >>> > > > > > > > >> > (requested 102400).
> >>> > > > > > > > >> > 2014-10-28 21:46:25,157 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Connected to
> >>> > > > 172.31.16.31:9092
> >>> > > > > > for
> >>> > > > > > > > >> > producing
> >>> > > > > > > > >> > 2014-10-28 21:46:25,160 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Disconnecting from
> >>> > > > > > > 172.31.16.31:9092
> >>> > > > > > > > >> > 2014-10-28 21:46:25,161 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.client.ClientUtils$-52: Successfully fetched
> >>> > metadata
> >>> > > > for
> >>> > > > > 1
> >>> > > > > > > > >> topic(s)
> >>> > > > > > > > >> > Set(Test)
> >>> > > > > > > > >> > 2014-10-28 21:46:25,161 pool-13-thread-1   WARN
> >>> > > > > > > > >> > kafka.producer.BrokerPartitionInfo-83: Error while
> >>> > fetching
> >>> > > > > > metadata
> >>> > > > > > > > >> > [{TopicMetadata for topic Test ->
> >>> > > > > > > > >> > No partition metadata for topic Test due to
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException}] for topic
> >>> > [Test]:
> >>> > > > > class
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException
> >>> > > > > > > > >> > 2014-10-28 21:46:25,161 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.producer.BrokerPartitionInfo-52: Getting
> broker
> >>> > > > partition
> >>> > > > > > info
> >>> > > > > > > > for
> >>> > > > > > > > >> > topic Test
> >>> > > > > > > > >> > 2014-10-28 21:46:25,162 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.client.ClientUtils$-68: Fetching metadata from
> >>> > broker
> >>> > > > > > > > >> > id:0,host:172.31.16.31,port:9092 with correlation id
> >>> 11
> >>> > for
> >>> > > 1
> >>> > > > > > > topic(s)
> >>> > > > > > > > >> > Set(Test)
> >>> > > > > > > > >> > 2014-10-28 21:46:25,162 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.network.BlockingChannel-52: Created socket
> with
> >>> > > > > SO_TIMEOUT =
> >>> > > > > > > > 10000
> >>> > > > > > > > >> > (requested 10000), SO_RCVBUF = 530904 (requested
> -1),
> >>> > > > SO_SNDBUF
> >>> > > > > =
> >>> > > > > > > > 102400
> >>> > > > > > > > >> > (requested 102400).
> >>> > > > > > > > >> > 2014-10-28 21:46:25,163 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Connected to
> >>> > > > 172.31.16.31:9092
> >>> > > > > > for
> >>> > > > > > > > >> > producing
> >>> > > > > > > > >> > 2014-10-28 21:46:25,166 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.SyncProducer-68: Disconnecting from
> >>> > > > > > > 172.31.16.31:9092
> >>> > > > > > > > >> > 2014-10-28 21:46:25,166 pool-13-thread-1  DEBUG
> >>> > > > > > > > >> > kafka.client.ClientUtils$-52: Successfully fetched
> >>> > metadata
> >>> > > > for
> >>> > > > > 1
> >>> > > > > > > > >> topic(s)
> >>> > > > > > > > >> > Set(Test)
> >>> > > > > > > > >> > 2014-10-28 21:46:25,166 pool-13-thread-1   WARN
> >>> > > > > > > > >> > kafka.producer.BrokerPartitionInfo-83: Error while
> >>> > fetching
> >>> > > > > > metadata
> >>> > > > > > > > >> > [{TopicMetadata for topic Test ->
> >>> > > > > > > > >> > No partition metadata for topic Test due to
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException}] for topic
> >>> > [Test]:
> >>> > > > > class
> >>> > > > > > > > >> > kafka.common.LeaderNotAvailableException
> >>> > > > > > > > >> > 2014-10-28 21:46:25,167 pool-13-thread-1  ERROR
> >>> > > > > > > > >> > kafka.producer.async.DefaultEventHandler-97: Failed
> to
> >>> > > collate
> >>> > > > > > > > messages
> >>> > > > > > > > >> by
> >>> > > > > > > > >> > topic, partition due to: Failed to fetch topic
> >>> metadata
> >>> > for
> >>> > > > > topic:
> >>> > > > > > > > Test
> >>> > > > > > > > >> > 2014-10-28 21:46:25,167 pool-13-thread-1   INFO
> >>> > > > > > > > >> > kafka.producer.async.DefaultEventHandler-68: Back
> off
> >>> for
> >>> > > 1000
> >>> > > > > ms
> >>> > > > > > > > before
> >>> > > > > > > > >> > retrying send. Remaining retries = 5
> >>> > > > > > > > >> >
> >>> > > > > > > > >> > As for /tmp/kafka-logs, I can see only these files -
> >>> > > > > > > > >> > recovery-point-offset-checkpoint
> >>> > > > replication-offset-checkpoint
> >>> > > > > > > > >> >
> >>> > > > > > > > >> > I do not see the usual Kafka log files here.
> >>> > > > > > > > >> >
> >>> > > > > > > > >> > thanks
> >>> > > > > > > > >> > Sameer
> >>> > > > > > > > >> >
> >>> > > > > > > > >> > On Tue, Oct 28, 2014 at 1:23 PM, Guozhang Wang <
> >>> > > > > > wangg...@gmail.com>
> >>> > > > > > > > >> wrote:
> >>> > > > > > > > >> >
> >>> > > > > > > > >> > > Sameer, could you check broker logs and see if it
> >>> has
> >>> > > > similar
> >>> > > > > > > > >> > > LeaderNotAvailable exceptions?
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > > Guozhang
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > > On Tue, Oct 28, 2014 at 10:32 AM, Sameer Yami <
> >>> > > > > sy...@ailive.net
> >>> > > > > > >
> >>> > > > > > > > >> wrote:
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > > > Yes. I can telnet from outside using the public
> >>> ip of
> >>> > > the
> >>> > > > > > > instance
> >>> > > > > > > > >> and
> >>> > > > > > > > >> > > port
> >>> > > > > > > > >> > > > of 9092
> >>> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > On Tue, Oct 28, 2014 at 10:08 AM, Roger Hoover <
> >>> > > > > > > > >> roger.hoo...@gmail.com
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > > > wrote:
> >>> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > > Just a guess but could it be a firewall issue?
> >>> Did
> >>> > > you
> >>> > > > > > enable
> >>> > > > > > > > >> > > > connections
> >>> > > > > > > > >> > > > > to port 9092 from outside EC2 in a security
> >>> group?
> >>> > > Can
> >>> > > > > you
> >>> > > > > > > > >> telnet to
> >>> > > > > > > > >> > > > each
> >>> > > > > > > > >> > > > > broker IP and port?
> >>> > > > > > > > >> > > > >
> >>> > > > > > > > >> > > > > On Tue, Oct 28, 2014 at 10:01 AM, Sameer Yami
> <
> >>> > > > > > > sy...@ailive.net
> >>> > > > > > > > >
> >>> > > > > > > > >> > > wrote:
> >>> > > > > > > > >> > > > >
> >>> > > > > > > > >> > > > > > There was a typo earlier.
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > > This is the output -
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > > Topic:Test PartitionCount:2
> >>> ReplicationFactor:1
> >>> > > > > > > Configs:
> >>> > > > > > > > >> > > > > >     Topic: Test        Partition: 0
> >>> Leader: 0
> >>> > > > > > > Replicas:
> >>> > > > > > > > 0
> >>> > > > > > > > >> > > > Isr: 0
> >>> > > > > > > > >> > > > > >     Topic: Test        Partition: 1
> >>> Leader: 0
> >>> > > > > > > Replicas:
> >>> > > > > > > > 0
> >>> > > > > > > > >> > > > Isr: 0
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > > thanks
> >>> > > > > > > > >> > > > > > Sameer
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > > On Tue, Oct 28, 2014 at 9:40 AM, Guozhang
> >>> Wang <
> >>> > > > > > > > >> wangg...@gmail.com
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > wrote:
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > > > Sameer,
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > The topic name is not consistent from your
> >>> > > describe
> >>> > > > > > > outputs.
> >>> > > > > > > > >> > > Normally
> >>> > > > > > > > >> > > > > > they
> >>> > > > > > > > >> > > > > > > should be either
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > >  Topic:*Test*    PartitionCount:2
> >>> > > > > ReplicationFactor:1
> >>> > > > > > > > >> > > Configs:
> >>> > > > > > > > >> > > > > > >     Topic: *Test*        Partition: 0
> >>> > Leader: 0
> >>> > > > > > > > >> Replicas: 0
> >>> > > > > > > > >> > > > > > Isr: 0
> >>> > > > > > > > >> > > > > > >     Topic: *Test*        Partition: 1
> >>> > Leader: 0
> >>> > > > > > > > >> Replicas: 0
> >>> > > > > > > > >> > > > > > Isr: 0
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > Or
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > >  Topic:*Heartbeat*    PartitionCount:2
> >>> > > > > > > > ReplicationFactor:1
> >>> > > > > > > > >> > > > > Configs:
> >>> > > > > > > > >> > > > > > >     Topic: *Heartbeat*        Partition: 0
> >>> > > > Leader:
> >>> > > > > 0
> >>> > > > > > > > >> > > Replicas:
> >>> > > > > > > > >> > > > 0
> >>> > > > > > > > >> > > > > > > Isr: 0
> >>> > > > > > > > >> > > > > > >     Topic: *Heartbeat*        Partition: 1
> >>> > > > Leader:
> >>> > > > > 0
> >>> > > > > > > > >> > > Replicas:
> >>> > > > > > > > >> > > > 0
> >>> > > > > > > > >> > > > > > > Isr: 0
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > Did you manually modify the topic name in
> >>> ZK?
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > Guozhang
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > On Tue, Oct 28, 2014 at 8:53 AM, Sameer
> >>> Yami <
> >>> > > > > > > > >> sy...@ailive.net>
> >>> > > > > > > > >> > > > wrote:
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > > Hi Guozhang,
> >>> > > > > > > > >> > > > > > > >
> >>> > > > > > > > >> > > > > > > > I was wondering if there is anything
> >>> wrong in
> >>> > > the
> >>> > > > > > above
> >>> > > > > > > > >> > > --describe
> >>> > > > > > > > >> > > > > > data.
> >>> > > > > > > > >> > > > > > > >
> >>> > > > > > > > >> > > > > > > > thanks
> >>> > > > > > > > >> > > > > > > >
> >>> > > > > > > > >> > > > > > > >
> >>> > > > > > > > >> > > > > > > > On Mon, Oct 27, 2014 at 2:56 PM, Sameer
> >>> Yami <
> >>> > > > > > > > >> sy...@ailive.net
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > wrote:
> >>> > > > > > > > >> > > > > > > >
> >>> > > > > > > > >> > > > > > > > > This is what I get -
> >>> > > > > > > > >> > > > > > > > >  bin/kafka-topics.sh --zookeeper
> >>> > > localhost:2181
> >>> > > > > > > > --describe
> >>> > > > > > > > >> > > > > > > > >  Topic:Heartbeat    PartitionCount:2
> >>> > > > > > > > >> ReplicationFactor:1
> >>> > > > > > > > >> > > > > > Configs:
> >>> > > > > > > > >> > > > > > > > >     Topic: Test        Partition: 0
> >>> > > Leader: 0
> >>> > > > > > > > >> > Replicas: 0
> >>> > > > > > > > >> > > > > > > Isr: 0
> >>> > > > > > > > >> > > > > > > > >     Topic: Test        Partition: 1
> >>> > > Leader: 0
> >>> > > > > > > > >> > Replicas: 0
> >>> > > > > > > > >> > > > > > > Isr: 0
> >>> > > > > > > > >> > > > > > > > >
> >>> > > > > > > > >> > > > > > > > > The topics are being created at
> >>> run-time.
> >>> > > > > > > > >> > > > > > > > >
> >>> > > > > > > > >> > > > > > > > > thanks
> >>> > > > > > > > >> > > > > > > > >
> >>> > > > > > > > >> > > > > > > > >
> >>> > > > > > > > >> > > > > > > > > On Sun, Oct 26, 2014 at 4:49 PM,
> >>> Guozhang
> >>> > > Wang <
> >>> > > > > > > > >> > > > wangg...@gmail.com
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > > > > > wrote:
> >>> > > > > > > > >> > > > > > > > >
> >>> > > > > > > > >> > > > > > > > >> Could you use the topic tools
> >>> "--describe"
> >>> > to
> >>> > > > > check
> >>> > > > > > > if
> >>> > > > > > > > >> the
> >>> > > > > > > > >> > > topic
> >>> > > > > > > > >> > > > > > > Test's
> >>> > > > > > > > >> > > > > > > > >> partitions leaders are available?
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > >
> >>> > > > > http://kafka.apache.org/documentation.html#basic_ops_add_topic
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >> Guozhang
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >> On Fri, Oct 24, 2014 at 2:43 PM,
> Sameer
> >>> > Yami
> >>> > > <
> >>> > > > > > > > >> > > sy...@ailive.net>
> >>> > > > > > > > >> > > > > > > wrote:
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >> > That is not the case.
> >>> > > > > > > > >> > > > > > > > >> > This detailed error is below. Due
> to
> >>> this
> >>> > > > > error ,
> >>> > > > > > > the
> >>> > > > > > > > >> > > producer
> >>> > > > > > > > >> > > > > is
> >>> > > > > > > > >> > > > > > > not
> >>> > > > > > > > >> > > > > > > > >> able
> >>> > > > > > > > >> > > > > > > > >> > to send messages
> >>> > > > > > > > >> > > > > > > > >> >
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,209 main  DEBUG
> >>> > > > > > > > >> > > > > > > > >>
> kafka.producer.BrokerPartitionInfo-52:
> >>> > > > > > > > >> > > > > > > > >> > Getting broker partition info for
> >>> topic
> >>> > > Test
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,209 main   INFO
> >>> > > > > > > > >> > > > > kafka.client.ClientUtils$-68:
> >>> > > > > > > > >> > > > > > > > >> Fetching
> >>> > > > > > > > >> > > > > > > > >> > metadata from broker
> >>> > > > > > > > id:0,host:54.215.200.61,port:9092
> >>> > > > > > > > >> > with
> >>> > > > > > > > >> > > > > > > > correlation
> >>> > > > > > > > >> > > > > > > > >> id
> >>> > > > > > > > >> > > > > > > > >> > 1 for 1 topic(s) Set(Test)
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,230 main  DEBUG
> >>> > > > > > > > >> > > > > > > kafka.network.BlockingChannel-52:
> >>> > > > > > > > >> > > > > > > > >> > Created socket with SO_TIMEOUT =
> >>> 10000
> >>> > > > > (requested
> >>> > > > > > > > >> 10000),
> >>> > > > > > > > >> > > > > > SO_RCVBUF
> >>> > > > > > > > >> > > > > > > =
> >>> > > > > > > > >> > > > > > > > >> > 187200 (requested -1), SO_SNDBUF =
> >>> 102400
> >>> > > > > > > (requested
> >>> > > > > > > > >> > > 102400).
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,231 main   INFO
> >>> > > > > > > > >> > > > > > kafka.producer.SyncProducer-68:
> >>> > > > > > > > >> > > > > > > > >> > Connected to 54.215.200.61:9092
> for
> >>> > > > producing
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,256 main   INFO
> >>> > > > > > > > >> > > > > > kafka.producer.SyncProducer-68:
> >>> > > > > > > > >> > > > > > > > >> > Disconnecting from
> >>> 54.215.200.61:9092
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,256 main  DEBUG
> >>> > > > > > > > >> > > > > kafka.client.ClientUtils$-52:
> >>> > > > > > > > >> > > > > > > > >> > Successfully fetched metadata for 1
> >>> > > topic(s)
> >>> > > > > > > > Set(Test)
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,257 main   WARN
> >>> > > > > > > > >> > > > > > > > >>
> kafka.producer.BrokerPartitionInfo-83:
> >>> > > > > > > > >> > > > > > > > >> > Error while fetching metadata
> >>> > > [{TopicMetadata
> >>> > > > > for
> >>> > > > > > > > topic
> >>> > > > > > > > >> > Test
> >>> > > > > > > > >> > > > ->
> >>> > > > > > > > >> > > > > > > > >> > No partition metadata for topic
> Test
> >>> due
> >>> > to
> >>> > > > > > > > >> > > > > > > > >> >
> >>> > kafka.common.LeaderNotAvailableException}]
> >>> > > > for
> >>> > > > > > > topic
> >>> > > > > > > > >> > [Test]:
> >>> > > > > > > > >> > > > > class
> >>> > > > > > > > >> > > > > > > > >> >
> >>> kafka.common.LeaderNotAvailableException
> >>> > > > > > > > >> > > > > > > > >> > 2014-10-24 14:31:47,258 main  ERROR
> >>> > > > > > > > >> > > > > > > > >> >
> >>> > > kafka.producer.async.DefaultEventHandler-97:
> >>> > > > > > Failed
> >>> > > > > > > > to
> >>> > > > > > > > >> > > collate
> >>> > > > > > > > >> > > > > > > > messages
> >>> > > > > > > > >> > > > > > > > >> by
> >>> > > > > > > > >> > > > > > > > >> > topic, partition due to: Failed to
> >>> fetch
> >>> > > > topic
> >>> > > > > > > > metadata
> >>> > > > > > > > >> > for
> >>> > > > > > > > >> > > > > topic:
> >>> > > > > > > > >> > > > > > > > Test
> >>> > > > > > > > >> > > > > > > > >> >
> >>> > > > > > > > >> > > > > > > > >> > On Fri, Oct 24, 2014 at 1:11 PM,
> >>> Guozhang
> >>> > > > Wang
> >>> > > > > <
> >>> > > > > > > > >> > > > > > wangg...@gmail.com>
> >>> > > > > > > > >> > > > > > > > >> wrote:
> >>> > > > > > > > >> > > > > > > > >> >
> >>> > > > > > > > >> > > > > > > > >> > > Could you see if this is the
> case:
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> >
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > >
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > >
> >>> > > > > > > > >> > > >
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> >
> >>> > > > > > > > >>
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whycan%27tmyconsumers/producersconnecttothebrokers
> >>> > > > > > > > >> > > > > > > > >> > > ?
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> > > On Fri, Oct 24, 2014 at 11:35 AM,
> >>> > Sameer
> >>> > > > > Yami <
> >>> > > > > > > > >> > > > > sy...@ailive.net
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > > >> wrote:
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> > > > Hi,
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > > > > > >> > > > We have a Kafka / Zookeeper
> test
> >>> > setup
> >>> > > > on a
> >>> > > > > > > > single
> >>> > > > > > > > >> EC2
> >>> > > > > > > > >> > > > > > machine.
> >>> > > > > > > > >> > > > > > > > The
> >>> > > > > > > > >> > > > > > > > >> > > > host.name is the private ip
> >>> address
> >>> > > and
> >>> > > > > the
> >>> > > > > > > > >> > > > > > > advertised.host.name
> >>> > > > > > > > >> > > > > > > > is
> >>> > > > > > > > >> > > > > > > > >> > the
> >>> > > > > > > > >> > > > > > > > >> > > > public address in
> >>> server.properties.
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > > > > > >> > > > We are trying to publish from
> >>> outside
> >>> > > EC2
> >>> > > > > > using
> >>> > > > > > > > the
> >>> > > > > > > > >> > > public
> >>> > > > > > > > >> > > > > ip
> >>> > > > > > > > >> > > > > > of
> >>> > > > > > > > >> > > > > > > > the
> >>> > > > > > > > >> > > > > > > > >> > EC2
> >>> > > > > > > > >> > > > > > > > >> > > > machine but get this error on
> the
> >>> > > client
> >>> > > > > > side -
> >>> > > > > > > > >> > > > > > > > >> > > > No partition metadata for topic
> >>> test
> >>> > > due
> >>> > > > to
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > > kafka.common.LeaderNotAvailableException}]
> >>> > > > > > for
> >>> > > > > > > > >> topic
> >>> > > > > > > > >> > > > [test]:
> >>> > > > > > > > >> > > > > > > class
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > kafka.common.LeaderNotAvailableException
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > > > > > >> > > > The zookeeper has the public ip
> >>> > stored
> >>> > > in
> >>> > > > > > > > >> > > /brokers/ids/0.
> >>> > > > > > > > >> > > > > > > > >> > > > We have tried multiple
> >>> combinations
> >>> > but
> >>> > > > it
> >>> > > > > > > works
> >>> > > > > > > > >> ONLY
> >>> > > > > > > > >> > > when
> >>> > > > > > > > >> > > > > the
> >>> > > > > > > > >> > > > > > > > >> > advertised
> >>> > > > > > > > >> > > > > > > > >> > > > host name is not used and the
> >>> client
> >>> > is
> >>> > > > > > > internal
> >>> > > > > > > > to
> >>> > > > > > > > >> > EC2.
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > > > > > >> > > > Please let me know what the
> >>> problem
> >>> > > could
> >>> > > > > be.
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > > > > > >> > > > thanks
> >>> > > > > > > > >> > > > > > > > >> > > > Sameer
> >>> > > > > > > > >> > > > > > > > >> > > >
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> > > --
> >>> > > > > > > > >> > > > > > > > >> > > -- Guozhang
> >>> > > > > > > > >> > > > > > > > >> > >
> >>> > > > > > > > >> > > > > > > > >> >
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >> --
> >>> > > > > > > > >> > > > > > > > >> -- Guozhang
> >>> > > > > > > > >> > > > > > > > >>
> >>> > > > > > > > >> > > > > > > > >
> >>> > > > > > > > >> > > > > > > > >
> >>> > > > > > > > >> > > > > > > >
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > > > --
> >>> > > > > > > > >> > > > > > > -- Guozhang
> >>> > > > > > > > >> > > > > > >
> >>> > > > > > > > >> > > > > >
> >>> > > > > > > > >> > > > >
> >>> > > > > > > > >> > > >
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> > > --
> >>> > > > > > > > >> > > -- Guozhang
> >>> > > > > > > > >> > >
> >>> > > > > > > > >> >
> >>> > > > > > > > >>
> >>> > > > > > > > >>
> >>> > > > > > > > >>
> >>> > > > > > > > >> --
> >>> > > > > > > > >> -- Guozhang
> >>> > > > > > > > >>
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > --
> >>> > > > > > > -- Guozhang
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > -- Guozhang
> >>> > > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > -- Guozhang
> >>> > >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> -- Guozhang
> >>>
> >>
> >>
> >
>



-- 
-- Guozhang

Re: Cannot connect to Kafka from outside of EC2

Reply via email to