ps here is the output the I add logging to the code
import logging
logging.basicConfig(
format='%(asctime)s.%(msecs)s:%(name)s:%(thread)d:%(levelname)s:%(process)d:%(message)s',
level=logging.DEBUG
)
2015-10-09
20:59:33,656.656.464099884:kafka.client:139860093155072:DEBUG:25285:Request
1: []
2015-10-09
20:59:33,657.657.119989395:kafka.conn:139860093155072:DEBUG:25285:Reinitializing
socket connection for 45.55.xxx.xxx:9092
2015-10-09
20:59:33,928.928.045034409:kafka.conn:139860093155072:DEBUG:25285:About to
send 30 bytes to Kafka, request 1
2015-10-09
20:59:33,928.928.556919098:kafka.conn:139860093155072:DEBUG:25285:Reading
response 1 from Kafka
2015-10-09
20:59:33,928.928.649902344:kafka.conn:139860093155072:DEBUG:25285:About to
read 4 bytes from Kafka
2015-10-09
20:59:34,184.184.187889099:kafka.conn:139860093155072:DEBUG:25285:Read 4/4
bytes from Kafka
2015-10-09
20:59:34,184.184.477090836:kafka.conn:139860093155072:DEBUG:25285:About to
read 12 bytes from Kafka
2015-10-09
20:59:34,184.184.587001801:kafka.conn:139860093155072:DEBUG:25285:Read
12/12 bytes from Kafka
2015-10-09
20:59:34,184.184.746026993:kafka.client:139860093155072:DEBUG:25285:Response
1: MetadataResponse(brokers=[], topics=[])
2015-10-09
20:59:34,184.184.829950333:kafka.client:139860093155072:DEBUG:25285:Updating
broker metadata: []
2015-10-09
20:59:34,184.184.88407135:kafka.client:139860093155072:DEBUG:25285:Updating
topic metadata: []
2015-10-09
20:59:34,185.185.06193161:kafka.client:139860093155072:DEBUG:25285:Request
2: ['topic-test-production']
2015-10-09
20:59:34,185.185.146093369:kafka.conn:139860093155072:DEBUG:25285:About to
send 53 bytes to Kafka, request 2
2015-10-09
20:59:34,185.185.309886932:kafka.conn:139860093155072:DEBUG:25285:Reading
response 2 from Kafka
2015-10-09
20:59:34,185.185.367107391:kafka.conn:139860093155072:DEBUG:25285:About to
read 4 bytes from Kafka
2015-10-09
20:59:34,445.445.18995285:kafka.conn:139860093155072:DEBUG:25285:Read 4/4
bytes from Kafka
2015-10-09
20:59:34,445.445.672988892:kafka.conn:139860093155072:DEBUG:25285:About to
read 41 bytes from Kafka
2015-10-09
20:59:34,445.445.739984512:kafka.conn:139860093155072:DEBUG:25285:Read
41/41 bytes from Kafka
2015-10-09
20:59:34,445.445.817947388:kafka.client:139860093155072:DEBUG:25285:Response
2: MetadataResponse(brokers=[],
topics=[TopicMetadata(topic='topic-test-production', error=5,
partitions=[])])
2015-10-09
20:59:34,445.445.895910263:kafka.client:139860093155072:DEBUG:25285:Updating
broker metadata: []
2015-10-09
20:59:34,445.445.94502449:kafka.client:139860093155072:DEBUG:25285:Updating
topic metadata: [TopicMetadata(topic='topic-test-production', error=5,
partitions=[])]
On Fri, Oct 9, 2015 at 7:00 PM, David Montgomery <davidmontgom...@gmail.com>
wrote:
> This is the python error
>
> kafka.common.LeaderNotAvailableError:
> TopicMetadata(topic='topic-test-production', error=5, partitions=[])
>
>
> On Fri, Oct 9, 2015 at 6:55 PM, David Montgomery <
> davidmontgom...@gmail.com> wrote:
>
>> Well i masked my true domain but the hostnames are valid and do work. I
>> do see the topics creates in ZK
>>
>>
>> brokers/topics/topic-test-production/partisions/0/state
>> {"controller_epoch":10,"leader":26665,"version":1,"leader_epoch":2,"isr":[26665]}
>>
>> brokers/topics/topic-test-production/partisions/1/state
>> {"controller_epoch":10,"leader":26665,"version":1,"leader_epoch":3,"isr":[26665]}
>> for example {"version":1,"partitions":{"1":[2028,26665],"0":[26665,2028]}}
>>
>> But now I do get the below on describe:
>> roduction PartitionCount:2 ReplicationFactor:2 Configs:
>> Topic: topic-test-production Partition: 0 Leader: 26665 Replicas:
>> 26665,2028 Isr: 26665
>> Topic: topic-test-production Partition: 1 Leader: 26665 Replicas:
>> 2028,26665 Isr: 26665
>>
>> But in kafka-python I still get this error
>> kafka.common.LeaderNotAvailableError:
>> TopicMetadata(topic='topic-test-production', error=5, partitions=[])
>>
>>
>>
>> On Fri, Oct 9, 2015 at 5:31 PM, Prabhjot Bharaj <prabhbha...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Is this a valid zookeeper connection string
>>>
>>>
>>> 1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
>>>
>>> Is 2.zk.do.production.test or 3.zk.do.production.test a valid hostname??
>>>
>>> Regards,
>>> Prabhjot
>>> On Oct 9, 2015 1:37 PM, "David Montgomery" <davidmontgom...@gmail.com>
>>> wrote:
>>>
>>> > PS I added one more server now 3 kafka servers. still kafka is not
>>> > working. Is there a known bug?
>>> >
>>> > On Fri, Oct 9, 2015 at 3:23 PM, David Montgomery <
>>> > davidmontgom...@gmail.com>
>>> > wrote:
>>> >
>>> > > i,
>>> > >
>>> > > I have 3 ZK servers and 2 kafka servers with 2 partitions.
>>> > >
>>> > > I am using kafka_2.11-0.8.2.1
>>> > >
>>> > > Here is how I am creating a topic:
>>> > >
>>> > > bin/kafka-topics.sh --zookeeper
>>> > >
>>> >
>>> 1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
>>> > > --create --topic topic-test-production --partitions 2
>>> > --replication-factor 2
>>> > >
>>> > > here is what I describe:
>>> > >
>>> > > bin/kafka-topics.sh
>>> > > --zookeeper
>>> >
>>> .zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
>>> > --describe
>>> > > --topic topic-test-production
>>> > >
>>> > > Topic:topic-test-production PartitionCount:2 ReplicationFactor:2
>>> Configs:
>>> > > Topic: topic-test-production Partition: 0 Leader: none Replicas:
>>> > > 26665,2028 Isr:
>>> > > Topic: topic-test-production Partition: 1 Leader: none Replicas:
>>> > > 2028,26665 Isr:
>>> > >
>>> > > When i try to write to kafka in python I get this error.
>>> > >
>>> > > kafka.common.LeaderNotAvailableError:
>>> > > TopicMetadata(topic='topic-test-production', error=5, partitions=[])
>>> > >
>>> > > What is wrong with kafka?
>>> > >
>>> > > Thanks
>>> > >
>>> > >
>>> > >
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>
>>
>
