Hi, Fathima, There is a known data loss issue that's described in KIP-101 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-101+-+Alter+Replication+Protocol+to+use+Leader+Epoch+rather+than+High+Watermark+for+Truncation). The issue happens rarely, but has been exposed in some of our system tests that mimic the testing that you have been doing. KIP-101 has been fixed in trunk and will be included in the next release (0.11.0.0) in June. Since then, we haven't observed similar data loss issue in our system tests.
If you want to test this out now, perhaps you could rerun your test in trunk. Thanks, Jun On Tue, May 16, 2017 at 2:09 AM, Fathima Amara <fath...@wso2.com> wrote: > > Hi all, > > I am using Kafka 2.11-0.10.0.1 and Zookeeper 3.4.8. > I have a cluster of 4 servers(A,B,C,D) running one kafka broker on each of > them and, one zookeeper server on server A. Data is initially produced from > server A using a Kafka Producer and it goes through servers B,C,D being > subjected to processing and finally reaches server A again(gets consumed > using a Kafka Consumer). > > Topics created on the end of each process has 2 partitions with a > replication-factor of 3. Other configurations include, > unclean.leader.election.enable=false > acks=all > retries=0 > I let the producer run for a while in server A, then kill one of the Kafka > brokers on the cluster(B,C,D) while data processing takes place and restart > it. When consuming from the end of server A, I notice a considerable amount > of data lost which varies on each run! ex:- on an input of 1 million events > 5930 events are lost. > > Is the reason for this the Kafka Producer not guaranteeing Exactly-once > processing or is this due to some other reason? what other reasons cause > data loss? >
