Ah, I think I see the confusion: Replicas don't actually ACK at all. What happens is that the replica manager waits for enough ISR replicas to reach the correct offset Partition.checkEnoughReplicasReachOffset(...) has this logic. A replica can't reach offset of second batch, without first having written the first batch. So I believe we are safe in this scenario.
Gwen On Tue, Jul 7, 2015 at 8:01 AM, Stevo Slavić <ssla...@gmail.com> wrote: > Hello Gwen, > > Thanks for fast response! > > Btw, congrats on officially becoming a Kafka committer and thanks, among > other things, for great "Intro to Kafka" video > http://shop.oreilly.com/product/0636920038603.do ! > > Have to read more docs and/or source. I thought this scenario is possible > because replica can fall behind (replica.lag.max.messages) and still be > considered ISR. Then I assumed also write can be ACKed by any ISR, and then > why not by one which has fallen more behind. > > Kind regards, > Stevo Slavic. > > On Tue, Jul 7, 2015 at 4:47 PM, Gwen Shapira <gshap...@cloudera.com> wrote: > >> I am not sure "different replica" can ACK the second back of messages >> while not having the first - from what I can see, it will need to be >> up-to-date on the latest messages (i.e. correct HWM) in order to ACK. >> >> On Tue, Jul 7, 2015 at 7:13 AM, Stevo Slavić <ssla...@gmail.com> wrote: >> > Hello Apache Kafka community, >> > >> > Documentation for min.insync.replicas in >> > http://kafka.apache.org/documentation.html#brokerconfigs states: >> > >> > "When used together, min.insync.replicas and request.required.acks allow >> > you to enforce greater durability guarantees. A typical scenario would be >> > to create a topic with a replication factor of 3, set min.insync.replicas >> > to 2, and produce with request.required.acks of -1. This will ensure that >> > the producer raises an exception if a majority of replicas do not >> receive a >> > write." >> > >> > Correct me if wrong (doc reference?), I assume min.insync.replicas >> includes >> > lead, so with min.insync.replicas=2, lead and one more replica besides >> lead >> > will have to ACK writes. >> > >> > In such setup, with minimalistic 3 brokers cluster, given that >> > - all 3 replicas are insync >> > - a batch of messages is written and ends up on lead and one replica ACKs >> > - another batch of messages ends up on lead and different replica ACKs >> > >> > Is it possible that when lead crashes, while replicas didn't catch up, >> > (part of) one batch of messages could be lost (since one replica becomes >> a >> > new lead, and it's only serving all reads and requests, and replication >> is >> > one way)? >> > >> > Kind regards, >> > Stevo Slavic. >>
