Re: kafka 0.8.2 broker behaviour

2016-05-24 Thread Anishek Agarwal
yeh mostly they are up always, sometimes when we do new deployment they might be down for about 15 mins or so. seems to have become slightly better now, i changed the replica socket buffer size and also the number of replica fetcher threads. On Mon, May 23, 2016 at 6:24 PM, Gerard Klijs wrote:

Re: kafka 0.8.2 broker behaviour

2016-05-23 Thread Gerard Klijs
Are you sure consumers are always up, when they are behind they could generate a lot of traffic in a small amount of time? On Mon, May 23, 2016 at 9:11 AM Anishek Agarwal wrote: > additionally all the read / writes are happening via storm topologies. > > On Mon, May 23, 2016 at 12:17 PM, Anishek

Re: kafka 0.8.2 broker behaviour

2016-05-23 Thread Anishek Agarwal
additionally all the read / writes are happening via storm topologies. On Mon, May 23, 2016 at 12:17 PM, Anishek Agarwal wrote: > Hello, > > we are using 4 kafka machines in production with 4 topics and each topic > either 16/32 partitions and replication factor of 2. each machine has 3 > disks