[
https://issues.apache.org/jira/browse/KAFKA-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733939#comment-14733939
]
Flavio Junqueira commented on KAFKA-2510:
-----------------------------------------
using a controlled shutdown in principle solves the problem because of the
following argument. let's assume that you have only two brokers to make it
simple. the directory is initially set to /a and the misconfiguration now
points each to /b. if you shutdown broker 1 and restarts it before doing it
with broker 2, then broker 1 will copy the data from broker 2.
the issue for me is that this is very fragile because there are all sort of
corner cases that you could have while doing this that could cause you to lose
data. say for example that you do not wait enough time and shutdown broker 2
before it has time to copy the data. in this case, you lost data.
> Prevent broker from re-replicating / losing data due to disk misconfiguration
> -----------------------------------------------------------------------------
>
> Key: KAFKA-2510
> URL: https://issues.apache.org/jira/browse/KAFKA-2510
> Project: Kafka
> Issue Type: Bug
> Reporter: Gwen Shapira
>
> Currently Kafka assumes that whatever it sees in the data directory is the
> correct state of the data.
> This means that if an admin mistakenly configures Chef to use wrong data
> directory, one of the following can happen:
> 1. The broker will replicate a bunch of partitions and take over the network
> 2. If you did this to enough brokers, you can lose entire topics and
> partitions.
> We have information about existing topics, partitions and their ISR in
> zookeeper.
> We need a mode in which if a broker starts, is in ISR for a partition and
> doesn't have any data or directory for the partition, the broker will issue a
> huge ERROR in the log and refuse to do anything for the partition.
> [~fpj] worked on the problem for ZK and had some ideas on what is required
> here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
