Re: Increase number of topic in Kafka leads zookeeper fail

2016-05-17 Thread Anas A
Hi there, Thank you for your valuable support now I changed the method as one topic with several partition for all devices and its solved my problem. *Thanks & Regards,* Anas A DBA, Trinity Mobility [image: facebook] [image: twitter]

Re: Increase number of topic in Kafka leads zookeeper fail

2016-05-17 Thread Tom Crayford
eason ... > > > > Btw ... +1 what Tom said > > > > Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat > > Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor > > Twitter : @ppatierno > > Linkedin : paolopatierno > > Blog : D

Re: Increase number of topic in Kafka leads zookeeper fail

2016-05-16 Thread Abhaya P
t it could be the reason ... > > Btw ... +1 what Tom said > > Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat > Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor > Twitter : @ppatierno > Linkedin : paolopatierno > Blog : DevExperience > > >

RE: Increase number of topic in Kafka leads zookeeper fail

2016-05-16 Thread Paolo Patierno
xperience > Date: Mon, 16 May 2016 05:26:31 -0700 > Subject: Re: Increase number of topic in Kafka leads zookeeper fail > From: christian.po...@gmail.com > To: users@kafka.apache.org > > +1 what Tom said. > > Curious though Anas, what motivated you to try a topic per device

Re: Increase number of topic in Kafka leads zookeeper fail

2016-05-16 Thread Christian Posta
+1 what Tom said. Curious though Anas, what motivated you to try a topic per device? was there something regarding management or security that you believe you can achieve with topic per device? On Mon, May 16, 2016 at 4:11 AM, Tom Crayford wrote: > Hi there, > > Generally you don't use a single

Re: Increase number of topic in Kafka leads zookeeper fail

2016-05-16 Thread Tom Crayford
Hi there, Generally you don't use a single topic per device in this use case, but one topic with some number of partitions and the key distribution based on device id. Kafka isn't designed for millions of low volume topics, but a few high volume ones. Thanks Tom Crayford Heroku Kafka On Mon, Ma

Increase number of topic in Kafka leads zookeeper fail

2016-05-15 Thread Anas A
We plan to use kafka as a message broker for IoT use case, where each device is considered as unique topic. when I simulated 10 message per second to 10 thousand topics zookeeper is getting bottle neck,all Kafka monitoring tools fails to read the throughput values and number of topics from JMX port