queue.enqueueTimeout.ms = -1 will block the producer instead of dropping messages. This might be useful if you have the producer being fed by scribe aggregators.
Thanks, Neha On Fri, Jan 4, 2013 at 9:04 AM, Raghu Angadi <raghu.ang...@gmail.com> wrote: > On Fri, Jan 4, 2013 at 8:39 AM, Jun Rao <jun...@gmail.com> wrote: > > > Do you know why some of the brokers are much slower than others? > > > We are currently running these in a shared environment, to make things > worse these machines have single spindle. We have to put up with that until > we move the brokers to dedicated hardware with multiple spindles. The > problem is a bit exaggerated in current setup. > > Even with dedicated hardware, I am expecting some variation. One slightly > degraded disk out of 12 could reduce effective b/w on all the spindles. > Unfortunately there will be occasional rack level network slowdowns that > take many hours to get fixed. > > In our case, we cannot let the back pressure from slow brokers propagate > upstream. Producers receive messages from scribe aggregators and just have > to drop the messages it they can't write fast enough. >
