Re: java consumer client sometimes work,but sometimes not.

2015-04-13 Thread François Méthot
It could be that the maximum number of connections was reached for client IP you are using. The defaut is 10. But it can be changed. I had similar intermittents issue because of that. The property is max.connections. per.ip Le 2015-04-12 11:20 PM, "kaybin wong" a écrit : > > hi there. > i got

Re: Number of Partitions and Performance

2015-04-07 Thread François Méthot
having > more > > or fewer partitions. > > > > -Jay > > > > On Tue, Apr 7, 2015 at 10:13 AM, François Méthot > > wrote: > > > > > Hi, > > > > > > We initially had configured our topics to have between 8 to 16 > >

Number of Partitions and Performance

2015-04-07 Thread François Méthot
Hi, We initially had configured our topics to have between 8 to 16 partitions each on a cluster of 10 brokers (vm with 2 cores, 16 MB ram, Few TB of SAN Disk). Then I came across the rule of thump formula *100 x b x r.* ( http://blog.confluent.io/2015/03/12/how-to-choose-the-number-of-topicspar

KafkaSpout forceFromStart Issue

2015-03-23 Thread François Méthot
Hi, We have a storm topology that uses Kafka to read a topic with 6 partitions. ( Kafka 0.8.2, Storm 0.9.3 ) Recently, we had to set the KafkaSpout to read from the beginning, so we temporary configured our KafkaConfig this way: kafkaConfig.forceFromStart=true kafkaConfig.startOffsetTime = Of

Kafka Offset Request for particular zookeeper chroot.

2015-02-26 Thread François Méthot
Hi, For sending Offset Request with Kafka 0.8.2, I used this example :Finding Starting Offset for Reads It works great on my test environment where a kafka broker uses the default zookeeper chroot (/). On another K