[ https://issues.apache.org/jira/browse/KAFKA-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946384#comment-15946384 ]
Stephane Maarek edited comment on KAFKA-3083 at 3/29/17 1:42 AM: ----------------------------------------------------------------- Just hit this issue in prod with Kafka 0.10.2.0, and the only solution was to reboot the broker that got put in an inconsistent state was (Author: stephane.maa...@gmail.com): Just hit this issue in prod, and the only solution was to reboot the broker that got put in an inconsistent state > a soft failure in controller may leave a topic partition in an inconsistent > state > --------------------------------------------------------------------------------- > > Key: KAFKA-3083 > URL: https://issues.apache.org/jira/browse/KAFKA-3083 > Project: Kafka > Issue Type: Bug > Components: core > Affects Versions: 0.9.0.0 > Reporter: Jun Rao > Assignee: Mayuresh Gharat > Labels: reliability > > The following sequence can happen. > 1. Broker A is the controller and is in the middle of processing a broker > change event. As part of this process, let's say it's about to shrink the isr > of a partition. > 2. Then broker A's session expires and broker B takes over as the new > controller. Broker B sends the initial leaderAndIsr request to all brokers. > 3. Broker A continues by shrinking the isr of the partition in ZK and sends > the new leaderAndIsr request to the broker (say C) that leads the partition. > Broker C will reject this leaderAndIsr since the request comes from a > controller with an older epoch. Now we could be in a situation that Broker C > thinks the isr has all replicas, but the isr stored in ZK is different. -- This message was sent by Atlassian JIRA (v6.3.15#6346)