RE: Re: Log.scala

2014-09-29 Thread Manjunath Shivakumar
dant. Guozhang On Fri, Sep 26, 2014 at 5:52 AM, Manjunath Shivakumar < manjunath.shivaku...@betfair.com> wrote: > Also, would this check be redundant because of > > https://github.com/apache/kafka/blob/7847e9c703f3a0b70519666cdb8a6e4c8e37c3a7/core/src/main/scala/kafka/log/Log

RE: Log.scala

2014-09-26 Thread Manjunath Shivakumar
Also, would this check be redundant because of https://github.com/apache/kafka/blob/7847e9c703f3a0b70519666cdb8a6e4c8e37c3a7/core/src/main/scala/kafka/log/Log.scala#L243 From: Manjunath Shivakumar [manjunath.shivaku...@betfair.com] Sent: 26 September 2014

RE: fsync behaviour

2014-08-21 Thread Manjunath Shivakumar
Or to re frame the question when replication is enabled. Will the message be available to consumers as soon as the message has been replicated to an in-sync follower, even before it is flushed to the disk. Thanks, Manju From: Manjunath Shivakumar Sent: 21 August

Re: fsync behaviour

2014-08-20 Thread Manjunath Shivakumar
disk before the response is sent to the producer. Thanks, Jun On Tue, Aug 19, 2014 at 6:00 PM, Manjunath Shivakumar < manjunath.shivaku...@betfair.com> wrote: > Hi, > > I wanted to understand the exact behaviour of kafka ( 0.8.1.1 ) w.r.t > fsync. I have read the documentatio

RE: Keep on getting kafka.common.OffsetOutOfRangeException: Random times

2014-08-20 Thread Manjunath Shivakumar
We had a similar issue in our dev environments, where we had to configure aggressive log retention to save space. And the clients kept failing with this error, on Mondays, because the message from friday had got deleted. Perhaps compaction would help in this scenario too? https://cwiki.apache.or

fsync behaviour

2014-08-19 Thread Manjunath Shivakumar
Hi, I wanted to understand the exact behaviour of kafka ( 0.8.1.1 ) w.r.t fsync. I have read the documentation and some related forum posts, but wanted to confirm the understanding here. 1) If we set the log flush to happen on every message, will it happen synchronously with each produce reque