[
https://issues.apache.org/jira/browse/KAFKA-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15468012#comment-15468012
]
Shikhar Bhushan commented on KAFKA-4127:
----------------------------------------
Dupe of KAFKA-3968
> Possible data loss
> ------------------
>
> Key: KAFKA-4127
> URL: https://issues.apache.org/jira/browse/KAFKA-4127
> Project: Kafka
> Issue Type: Bug
> Environment: Normal three node Kafka cluster. All machines running
> linux.
> Reporter: Ramnatthan Alagappan
>
> I am running a three node Kakfa cluster. ZooKeeper runs in a standalone mode.
> When I create a new message topic, I see the following sequence of system
> calls:
> mkdir("/appdir/my-topic1-0")
> creat("/appdir/my-topic1-0/00000000000000000000.log")
> I have configured Kafka to write the messages persistently to the disk before
> acknowledging the client. Specifically, I have set flush.interval_messages to
> 1, min_insync_replicas to 3, and disabled dirty election. Now, I insert a
> new message into the created topic.
> I see that Kafka writes the message to the log file and flushes the data down
> to disk by carefully fsync'ing the log file. I get an acknowledgment back
> from the cluster after the message is safely persisted on all three replicas
> and written to disk.
> Unfortunately, Kafka can still lose data since it does not explicitly fsync
> the directory to persist the directory entries of the topic directory and the
> log file. If a crash happens after acknowledging the client, it is possible
> for Kafka lose the directory entry for the topic directory or the log file.
> Many systems carefully issue fsync to the parent directory when a new file or
> directory is created. This is required for the file to be completely
> persisted to the disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
