from kafka import SimpleProducer
from kafka import KafkaClient
kafka_host_list =
['192.241.xxx.xxx:9092','45.55.xxx.xxx:9092','107.170.xxx.xxx:9092']
kafka = KafkaClient(kafka_host_list)
producer = SimpleProducer(kafka, async=False)
test_topic = 'topic-test-production'
test_payload = 'test'
print producer.send_messages(test_topic,test_payload)
Traceback (most recent call last):
File
"/home/ubuntu/workspace/feed-tests/tests/post_druid_feeds_kafka_write.py",
line 9, in <module>
print producer.send_messages(test_topic,test_payload)
File
"/usr/local/lib/python2.7/dist-packages/kafka_python-0.9.5_dev-py2.7.egg/kafka/producer/simple.py",
line 52, in send_messages
partition = self._next_partition(topic)
File
"/usr/local/lib/python2.7/dist-packages/kafka_python-0.9.5_dev-py2.7.egg/kafka/producer/simple.py",
line 36, in _next_partition
self.client.load_metadata_for_topics(topic)
File
"/usr/local/lib/python2.7/dist-packages/kafka_python-0.9.5_dev-py2.7.egg/kafka/client.py",
line 383, in load_metadata_for_topics
kafka.common.check_error(topic_metadata)
File
"/usr/local/lib/python2.7/dist-packages/kafka_python-0.9.5_dev-py2.7.egg/kafka/common.py",
line 233, in check_error
raise error_class(response)
kafka.common.LeaderNotAvailableError:
TopicMetadata(topic='topic-test-production', error=5, partitions=[])
bin/kafka-topics.sh --zookeeper ... --describe --topic
topic-test-production
Topic:topic-test-production PartitionCount:2 ReplicationFactor:2 Configs:
Topic: topic-test-production Partition: 0 Leader: 26665 Replicas:
26665,2028 Isr:
26665
Topic: topic-test-production Partition: 1 Leader: 26665 Replicas:
2028,26665 Isr:
26665
On Fri, Oct 9, 2015 at 9:03 PM, Prabhjot Bharaj <prabhbha...@gmail.com>
wrote:
> Hi,
>
> Please share the code snippet for Kafka python
>
> Regards,
> Prabhjot
> On Oct 9, 2015 4:30 PM, "David Montgomery" <davidmontgom...@gmail.com>
> wrote:
>
> > This is the python error
> >
> > kafka.common.LeaderNotAvailableError:
> > TopicMetadata(topic='topic-test-production', error=5, partitions=[])
> >
> >
> > On Fri, Oct 9, 2015 at 6:55 PM, David Montgomery <
> > davidmontgom...@gmail.com>
> > wrote:
> >
> > > Well i masked my true domain but the hostnames are valid and do work.
> I
> > > do see the topics creates in ZK
> > >
> > >
> > > brokers/topics/topic-test-production/partisions/0/state
> >
> {"controller_epoch":10,"leader":26665,"version":1,"leader_epoch":2,"isr":[26665]}
> > >
> > > brokers/topics/topic-test-production/partisions/1/state
> >
> {"controller_epoch":10,"leader":26665,"version":1,"leader_epoch":3,"isr":[26665]}
> > > for example
> > {"version":1,"partitions":{"1":[2028,26665],"0":[26665,2028]}}
> > >
> > > But now I do get the below on describe:
> > > roduction PartitionCount:2 ReplicationFactor:2 Configs:
> > > Topic: topic-test-production Partition: 0 Leader: 26665 Replicas:
> > > 26665,2028 Isr: 26665
> > > Topic: topic-test-production Partition: 1 Leader: 26665 Replicas:
> > > 2028,26665 Isr: 26665
> > >
> > > But in kafka-python I still get this error
> > > kafka.common.LeaderNotAvailableError:
> > > TopicMetadata(topic='topic-test-production', error=5, partitions=[])
> > >
> > >
> > >
> > > On Fri, Oct 9, 2015 at 5:31 PM, Prabhjot Bharaj <prabhbha...@gmail.com
> >
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> Is this a valid zookeeper connection string
> > >>
> > >>
> > >>
> >
> 1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
> > >>
> > >> Is 2.zk.do.production.test or 3.zk.do.production.test a valid
> hostname??
> > >>
> > >> Regards,
> > >> Prabhjot
> > >> On Oct 9, 2015 1:37 PM, "David Montgomery" <davidmontgom...@gmail.com
> >
> > >> wrote:
> > >>
> > >> > PS I added one more server now 3 kafka servers. still kafka is not
> > >> > working. Is there a known bug?
> > >> >
> > >> > On Fri, Oct 9, 2015 at 3:23 PM, David Montgomery <
> > >> > davidmontgom...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > i,
> > >> > >
> > >> > > I have 3 ZK servers and 2 kafka servers with 2 partitions.
> > >> > >
> > >> > > I am using kafka_2.11-0.8.2.1
> > >> > >
> > >> > > Here is how I am creating a topic:
> > >> > >
> > >> > > bin/kafka-topics.sh --zookeeper
> > >> > >
> > >> >
> > >>
> >
> 1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
> > >> > > --create --topic topic-test-production --partitions 2
> > >> > --replication-factor 2
> > >> > >
> > >> > > here is what I describe:
> > >> > >
> > >> > > bin/kafka-topics.sh
> > >> > > --zookeeper
> > >> >
> > >>
> >
> .zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
> > >> > --describe
> > >> > > --topic topic-test-production
> > >> > >
> > >> > > Topic:topic-test-production PartitionCount:2 ReplicationFactor:2
> > >> Configs:
> > >> > > Topic: topic-test-production Partition: 0 Leader: none Replicas:
> > >> > > 26665,2028 Isr:
> > >> > > Topic: topic-test-production Partition: 1 Leader: none Replicas:
> > >> > > 2028,26665 Isr:
> > >> > >
> > >> > > When i try to write to kafka in python I get this error.
> > >> > >
> > >> > > kafka.common.LeaderNotAvailableError:
> > >> > > TopicMetadata(topic='topic-test-production', error=5,
> partitions=[])
> > >> > >
> > >> > > What is wrong with kafka?
> > >> > >
> > >> > > Thanks
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
