Re: mirrormaker's configuration to minimize/prevent data loss

2014-06-16 Thread Jun Rao
Currently, mirrormaker only logs the error if the producer fails. You can potentially increase # retries to deal with producer failures. Thanks, Jun On Mon, Jun 16, 2014 at 3:53 PM, Andrey Yegorov wrote: > As I read, consumer and producer in mirrormaker are independent and use > queue to comm

mirrormaker's configuration to minimize/prevent data loss

2014-06-16 Thread Andrey Yegorov
As I read, consumer and producer in mirrormaker are independent and use queue to communicate. Therefore consumers keep on consuming/commiting offsets to zk even if producer is failing. Is it still the way it works in 0.8.0, any plans to change? Is there any way to minimize data loss in this case?