Scalability with the Number of Partitions

2016-05-19 Thread Yazeed Alabdulkarim
es without any problem but I am worried that it maybe impacting the performance. I tried to set all the config timeout parameters to a large number. Both Kafka consumers and server are running on the same machine. -- Best Regards, Yazeed Alabdulkarim

Re: Scalability with the Number of Partitions

2016-05-19 Thread Yazeed Alabdulkarim
ge seems pretty bad - it sounds like the broker is falling > over repeatedly. I'd check the broker logs and metrics (see > http://docs.confluent.io/2.0.1/kafka/monitoring.html). > > Thanks > > Tom Crayford > Heroku Kafka > > On Thu, May 19, 2016 at 7:14 PM, Yazeed Alab

Kafka Scalability with the Number of Partitions

2016-05-23 Thread Yazeed Alabdulkarim
duration of the test reaches a steady state. I am monitoring the resources of my server and didn't see any bottleneck. Am I missing something here? Shouldn't Kafka consumers scale with the number of partitions? -- Best Regards, Yazeed Alabdulkarim

Re: Kafka Scalability with the Number of Partitions

2016-05-25 Thread Yazeed Alabdulkarim
nk a lot for your help. I really appreciate it. On Tue, May 24, 2016 at 7:51 AM, Tom Crayford wrote: > What's your server setup for the brokers and consumers? Generally I'd > expect something to be exhausted here and that to end up being the > bottleneck. > > Thanks > >