gautham varada created KAFKA-1633:
-------------------------------------
Summary: Data loss if broker is killed
Key: KAFKA-1633
URL: https://issues.apache.org/jira/browse/KAFKA-1633
Project: Kafka
Issue Type: Bug
Components: producer
Affects Versions: 0.8.1.1
Environment: centos 6.3, open jdk 7
Reporter: gautham varada
Assignee: Jun Rao
We have a 2 node kafka cluster, we experienced data loss when we did a kill -9
on the brokers. We also found a work around to prevent this loss.
Replication factor :2, 4 partitions
Steps to reproduce
1. Create a 2 node cluster with replication factor 2, num partitions 4
2. We used Jmeter to pump events
3. We used kafka web console to inspect the log size after the test
During the test, we simultaneously killed the brokers using kill -9 and we
tallied the metrics reported by jmeter and the size we observed in the web
console, we lost tons of messages.
We went back and set the Producer retry to 1 instead of the default 3 and
repeated the above tests and we did not loose a single message.
We repeated the above tests with the Producer retry set to 3 and 1 with a
single broker and we observed data loss when the retry was 3 and no loss when
the retry was 1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
