Hi,
Your metadata response is empty. Here is an output for my kafka cluster:
*>>> kafka =
KafkaClient(['96.a.a.a:9092','96.b.b.b:9092','96.c.c.c:9092','96.d.d.d:9092','96.e.e.e:9092'])*
*2015-10-09
21:16:53,575.575.392961502:kafka.client:140735282348800:DEBUG:8705:Request
1: []*
*2015-10-09
21:16:53,580.580.756902695:kafka.conn:140735282348800:DEBUG:8705:Reinitializing
socket connection for 96.b.b.b:9092*
*2015-10-09
21:16:53,981.981.142044067:kafka.conn:140735282348800:DEBUG:8705:About to
send 30 bytes to Kafka, request 1*
*2015-10-09
21:16:53,981.981.415987015:kafka.conn:140735282348800:DEBUG:8705:Reading
response 1 from Kafka*
*2015-10-09
21:16:53,981.981.508016586:kafka.conn:140735282348800:DEBUG:8705:About to
read 4 bytes from Kafka*
*2015-10-09
21:16:54,596.596.358060837:kafka.conn:140735282348800:DEBUG:8705:Read 4/4
bytes from Kafka*
*2015-10-09
21:16:54,596.596.518039703:kafka.conn:140735282348800:DEBUG:8705:About to
read 389 bytes from Kafka*
*2015-10-09
21:16:54,596.596.612930298:kafka.conn:140735282348800:DEBUG:8705:Read
389/389 bytes from Kafka*
*2015-10-09
21:16:54,596.596.745014191:kafka.client:140735282348800:DEBUG:8705:Response
1: MetadataResponse(brokers=[BrokerMetadata(nodeId=1, host='96.a.a.a.a',
port=9092), BrokerMetadata(nodeId=6, host='96.d.d.d', port=9092),
BrokerMetadata(nodeId=2, host='96.b.b.b', port=9092),
BrokerMetadata(nodeId=7, host='96.e.e.e', port=9092),
BrokerMetadata(nodeId=4, host='96.c.c.c', port=9092)],
topics=[TopicMetadata(topic='part_1_repl_3', error=0,
partitions=[PartitionMetadata(topic='part_1_repl_3', partition=0, leader=2,
replicas=(2, 6, 7), isr=(2, 6, 7), error=0)]),
TopicMetadata(topic='part_1_repl_3_4', error=0,
partitions=[PartitionMetadata(topic='part_1_repl_3_4', partition=0,
leader=6, replicas=(6,), isr=(6,), error=0)])])*
*2015-10-09
21:16:54,596.596.867084503:kafka.client:140735282348800:INFO:8705:Updating
broker metadata: [BrokerMetadata(nodeId=1, host='96.a.a.a.a', port=9092),
BrokerMetadata(nodeId=6, host='96.d.d.d', port=9092),
BrokerMetadata(nodeId=2, host='96.b.b.b', port=9092),
BrokerMetadata(nodeId=7, host='96.e.e.e', port=9092),
BrokerMetadata(nodeId=4, host='96.c.c.c', port=9092)] *
*2015-10-09
21:16:54,596.596.991062164:kafka.client:140735282348800:INFO:8705:Updating
topic metadata: [TopicMetadata(topic='part_1_repl_3', error=0,
partitions=[PartitionMetadata(topic='part_1_repl_3', partition=0, leader=2,
replicas=(2, 6, 7), isr=(2, 6, 7), error=0)]),
TopicMetadata(topic='part_1_repl_3_4', error=0,
partitions=[PartitionMetadata(topic='part_1_repl_3_4', partition=0,
leader=6, replicas=(6,), isr=(6,), error=0)])]*
Is your cluster up ?
Are you behind some firewall ?
By the way, I'm using kafka-python-0.9.4 from my mac
Regards,
Prabhjot
On Fri, Oct 9, 2015 at 7:32 PM, David Montgomery <davidmontgom...@gmail.com>
wrote:
> ps here is the output the I add logging to the code
>
> import logging
> logging.basicConfig(
>
>
> format='%(asctime)s.%(msecs)s:%(name)s:%(thread)d:%(levelname)s:%(process)d:%(message)s',
> level=logging.DEBUG
> )
>
> 2015-10-09
> 20:59:33,656.656.464099884:kafka.client:139860093155072:DEBUG:25285:Request
> 1: []
> 2015-10-09
>
> 20:59:33,657.657.119989395:kafka.conn:139860093155072:DEBUG:25285:Reinitializing
> socket connection for 45.55.xxx.xxx:9092
> 2015-10-09
> 20:59:33,928.928.045034409:kafka.conn:139860093155072:DEBUG:25285:About to
> send 30 bytes to Kafka, request 1
> 2015-10-09
> 20:59:33,928.928.556919098:kafka.conn:139860093155072:DEBUG:25285:Reading
> response 1 from Kafka
> 2015-10-09
> 20:59:33,928.928.649902344:kafka.conn:139860093155072:DEBUG:25285:About to
> read 4 bytes from Kafka
> 2015-10-09
> 20:59:34,184.184.187889099:kafka.conn:139860093155072:DEBUG:25285:Read 4/4
> bytes from Kafka
> 2015-10-09
> 20:59:34,184.184.477090836:kafka.conn:139860093155072:DEBUG:25285:About to
> read 12 bytes from Kafka
> 2015-10-09
> 20:59:34,184.184.587001801:kafka.conn:139860093155072:DEBUG:25285:Read
> 12/12 bytes from Kafka
> 2015-10-09
>
> 20:59:34,184.184.746026993:kafka.client:139860093155072:DEBUG:25285:Response
> 1: MetadataResponse(brokers=[], topics=[])
> 2015-10-09
>
> 20:59:34,184.184.829950333:kafka.client:139860093155072:DEBUG:25285:Updating
> broker metadata: []
> 2015-10-09
> 20:59:34,184.184.88407135:kafka.client:139860093155072:DEBUG:25285:Updating
> topic metadata: []
> 2015-10-09
> 20:59:34,185.185.06193161:kafka.client:139860093155072:DEBUG:25285:Request
> 2: ['topic-test-production']
> 2015-10-09
> 20:59:34,185.185.146093369:kafka.conn:139860093155072:DEBUG:25285:About to
> send 53 bytes to Kafka, request 2
> 2015-10-09
> 20:59:34,185.185.309886932:kafka.conn:139860093155072:DEBUG:25285:Reading
> response 2 from Kafka
> 2015-10-09
> 20:59:34,185.185.367107391:kafka.conn:139860093155072:DEBUG:25285:About to
> read 4 bytes from Kafka
> 2015-10-09
> 20:59:34,445.445.18995285:kafka.conn:139860093155072:DEBUG:25285:Read 4/4
> bytes from Kafka
> 2015-10-09
> 20:59:34,445.445.672988892:kafka.conn:139860093155072:DEBUG:25285:About to
> read 41 bytes from Kafka
> 2015-10-09
> 20:59:34,445.445.739984512:kafka.conn:139860093155072:DEBUG:25285:Read
> 41/41 bytes from Kafka
> 2015-10-09
>
> 20:59:34,445.445.817947388:kafka.client:139860093155072:DEBUG:25285:Response
> 2: MetadataResponse(brokers=[],
> topics=[TopicMetadata(topic='topic-test-production', error=5,
> partitions=[])])
> 2015-10-09
>
> 20:59:34,445.445.895910263:kafka.client:139860093155072:DEBUG:25285:Updating
> broker metadata: []
> 2015-10-09
> 20:59:34,445.445.94502449:kafka.client:139860093155072:DEBUG:25285:Updating
> topic metadata: [TopicMetadata(topic='topic-test-production', error=5,
> partitions=[])]
>
>
> On Fri, Oct 9, 2015 at 7:00 PM, David Montgomery <
> davidmontgom...@gmail.com>
> wrote:
>
> > This is the python error
> >
> > kafka.common.LeaderNotAvailableError:
> > TopicMetadata(topic='topic-test-production', error=5, partitions=[])
> >
> >
> > On Fri, Oct 9, 2015 at 6:55 PM, David Montgomery <
> > davidmontgom...@gmail.com> wrote:
> >
> >> Well i masked my true domain but the hostnames are valid and do work. I
> >> do see the topics creates in ZK
> >>
> >>
> >> brokers/topics/topic-test-production/partisions/0/state
> {"controller_epoch":10,"leader":26665,"version":1,"leader_epoch":2,"isr":[26665]}
> >>
> >> brokers/topics/topic-test-production/partisions/1/state
> {"controller_epoch":10,"leader":26665,"version":1,"leader_epoch":3,"isr":[26665]}
> >> for example
> {"version":1,"partitions":{"1":[2028,26665],"0":[26665,2028]}}
> >>
> >> But now I do get the below on describe:
> >> roduction PartitionCount:2 ReplicationFactor:2 Configs:
> >> Topic: topic-test-production Partition: 0 Leader: 26665 Replicas:
> >> 26665,2028 Isr: 26665
> >> Topic: topic-test-production Partition: 1 Leader: 26665 Replicas:
> >> 2028,26665 Isr: 26665
> >>
> >> But in kafka-python I still get this error
> >> kafka.common.LeaderNotAvailableError:
> >> TopicMetadata(topic='topic-test-production', error=5, partitions=[])
> >>
> >>
> >>
> >> On Fri, Oct 9, 2015 at 5:31 PM, Prabhjot Bharaj <prabhbha...@gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Is this a valid zookeeper connection string
> >>>
> >>>
> >>>
> 1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
> >>>
> >>> Is 2.zk.do.production.test or 3.zk.do.production.test a valid
> hostname??
> >>>
> >>> Regards,
> >>> Prabhjot
> >>> On Oct 9, 2015 1:37 PM, "David Montgomery" <davidmontgom...@gmail.com>
> >>> wrote:
> >>>
> >>> > PS I added one more server now 3 kafka servers. still kafka is not
> >>> > working. Is there a known bug?
> >>> >
> >>> > On Fri, Oct 9, 2015 at 3:23 PM, David Montgomery <
> >>> > davidmontgom...@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > i,
> >>> > >
> >>> > > I have 3 ZK servers and 2 kafka servers with 2 partitions.
> >>> > >
> >>> > > I am using kafka_2.11-0.8.2.1
> >>> > >
> >>> > > Here is how I am creating a topic:
> >>> > >
> >>> > > bin/kafka-topics.sh --zookeeper
> >>> > >
> >>> >
> >>>
> 1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
> >>> > > --create --topic topic-test-production --partitions 2
> >>> > --replication-factor 2
> >>> > >
> >>> > > here is what I describe:
> >>> > >
> >>> > > bin/kafka-topics.sh
> >>> > > --zookeeper
> >>> >
> >>>
> .zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
> >>> > --describe
> >>> > > --topic topic-test-production
> >>> > >
> >>> > > Topic:topic-test-production PartitionCount:2 ReplicationFactor:2
> >>> Configs:
> >>> > > Topic: topic-test-production Partition: 0 Leader: none Replicas:
> >>> > > 26665,2028 Isr:
> >>> > > Topic: topic-test-production Partition: 1 Leader: none Replicas:
> >>> > > 2028,26665 Isr:
> >>> > >
> >>> > > When i try to write to kafka in python I get this error.
> >>> > >
> >>> > > kafka.common.LeaderNotAvailableError:
> >>> > > TopicMetadata(topic='topic-test-production', error=5,
> partitions=[])
> >>> > >
> >>> > > What is wrong with kafka?
> >>> > >
> >>> > > Thanks
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>
--
---------------------------------------------------------
"There are only 10 types of people in the world: Those who understand
binary, and those who don't"
