Re: High level consumer Blocked when there is still message in topic

2013-11-13 Thread Joel Koshy
On Wed, Nov 13, 2013 at 11:54:07AM -0800, hsy...@gmail.com wrote: > Since you have a cluster, why not distribute the consumers in different > nodes instead of threads. I think that's the only way to scale up with > kafka. Depending on your CPU-specs you should be able to add threads to scale out -

Re: High level consumer Blocked when there is still message in topic

2013-11-13 Thread hsy...@gmail.com
Since you have a cluster, why not distribute the consumers in different nodes instead of threads. I think that's the only way to scale up with kafka. Question here: if there are more and more high-level consumers, is there a bottleneck on the zookeeper? On Tue, Nov 12, 2013 at 9:27 PM, Jun Rao w

Re: High level consumer Blocked when there is still message in topic

2013-11-12 Thread Jun Rao
What's the max lag (reported in JMX) in the consumer? Can the consumer keep up with the incoming data rate? Thanks, Jun On Tue, Nov 12, 2013 at 7:19 PM, 李帅 wrote: > Hi, > >I use Kafka 0.8 high level consumer reads message from topic > stream, 3 replica and 10 paritions. > >When I use