I didn't get any hits on this on users@, so I'm forwarding this to dev@. Any explanation for this would be greatly appreciated!
---------- Forwarded message ---------- From: Nick Travers <n.e.trav...@gmail.com> Date: Sat, Feb 18, 2017 at 5:04 PM Subject: Producer acks=1, clean broker shutdown and data loss To: us...@kafka.apache.org Hi - I'm trying to understand the expected behavior of the scenario in which I have a producer with `acks=1` (i.e. partition leader acks only) and I cleanly shut down a broker (via `KafkaServer#shutdown`). I am running my test scenario with three brokers (0.10.1.1), with a default replication count and default partition count of three. The producer (also 0.10.1.1) is writing to a single topic. When I shut down one broker, I observe that in some instances there is data loss on the topic that the producer was sending to, around the instant where the broker was shut down and the producer sees that a new leader for one of the partitions has been re-elected. Reading the documentation [0], I see the following for the `acks=1` config: This will mean the leader will write the record to its local log but will > respond without awaiting full acknowledgement from all followers. In this > case should the leader fail immediately after acknowledging the record but > before the followers have replicated it then the record will be lost. While it is clear to me that in the case of a _failure_ of this broker that messages will be lost (as they have not yet been replicated to a follower), it isn't clear to me what the expected behavior is in the case where this broker is told to _cleanly_ shut down. I understand that having a setup where `replicas=3`, `min.insync.replicas=2` and `acks=-1` is much safer (and I've verified this works as expected for the same test scenario), I'd like to understand the semantics of the `acks=1` case nonetheless. Thanks in advance. - nick [0]: https://github.com/apache/kafka/blob/0.10.1.1/clients/ src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java#L86
