Re: Large kafka deployment on virtual hardware

2016-05-24 Thread Tom Crayford
d replica lag to be the biggest issue. Before we lost our > metrics I noticed a lot of leader change activity - could this be a symptom > of the offline partitions? > > Kind regards, > > Jahn Roux > > > -Original Message- > From: Tom Crayford [mailto:tcrayf...@her

RE: Large kafka deployment on virtual hardware

2016-05-24 Thread Jahn Roux
d a lot of leader change activity - could this be a symptom of the offline partitions? Kind regards, Jahn Roux -Original Message- From: Tom Crayford [mailto:tcrayf...@heroku.com] Sent: Tuesday, May 24, 2016 3:07 PM To: Users Subject: Re: Large kafka deployment on virtual hardware Jahn, Are

Re: Large kafka deployment on virtual hardware

2016-05-24 Thread Tom Crayford
Original Message- > From: Sharninder [mailto:sharnin...@gmail.com] > Sent: Tuesday, May 24, 2016 10:49 AM > To: users@kafka.apache.org > Subject: Re: Large kafka deployment on virtual hardware > > I'm sure you checked this but since these are virtual machines, is it > possib

RE: Large kafka deployment on virtual hardware

2016-05-24 Thread Jahn Roux
Roux -Original Message- From: Sharninder [mailto:sharnin...@gmail.com] Sent: Tuesday, May 24, 2016 10:49 AM To: users@kafka.apache.org Subject: Re: Large kafka deployment on virtual hardware I'm sure you checked this but since these are virtual machines, is it possible there is

Re: Large kafka deployment on virtual hardware

2016-05-24 Thread Sharninder
I'm sure you checked this but since these are virtual machines, is it possible there is just contention for resources? Network clogged or some other simpler explanation like that? On Mon, May 23, 2016 at 9:42 PM, Jahn Roux wrote: > I have a large Kafka deployment on virtual hardware: 120 brokers

Re: Large kafka deployment on virtual hardware

2016-05-24 Thread Ben Davison
gt; From: Jens Rantil [mailto:jens.ran...@tink.se] > Sent: Monday, May 23, 2016 11:00 PM > To: users@kafka.apache.org > Subject: Re: Large kafka deployment on virtual hardware > > Hi Jahn, > > How is the load on Zookeeper? How often are you committing your offsets? > Could that be

RE: Large kafka deployment on virtual hardware

2016-05-24 Thread Jahn Roux
[mailto:jens.ran...@tink.se] Sent: Monday, May 23, 2016 11:00 PM To: users@kafka.apache.org Subject: Re: Large kafka deployment on virtual hardware Hi Jahn, How is the load on Zookeeper? How often are you committing your offsets? Could that be an issue? Cheers, Jens Den mån 23 maj 2016 18:12Jahn

Re: Large kafka deployment on virtual hardware

2016-05-23 Thread Jens Rantil
Hi Jahn, How is the load on Zookeeper? How often are you committing your offsets? Could that be an issue? Cheers, Jens Den mån 23 maj 2016 18:12Jahn Roux skrev: > I have a large Kafka deployment on virtual hardware: 120 brokers on 32gb > memory 8 core virtual machines. Gigabit network, RHEL 6.