Was there an answer for 0.8.1 getting stuck in preferred leader election? I'm seeing this as well. Is there a JIRA ticket on this issue?
On Fri, Mar 21, 2014 at 1:15 PM, Ryan Berdeen <rberd...@hubspot.com> wrote: > So, for 0.8 without "controlled.shutdown.enable", why would ShutdownBroker > and restarting cause under-replication and producer exceptions? How can I > upgrade gracefully? > > What's up with 0.8.1 getting stuck in preferred leader election? > > > On Fri, Mar 21, 2014 at 12:18 AM, Neha Narkhede <neha.narkh...@gmail.com > >wrote: > > > Which brings up the question - Do we need ShutdownBroker anymore? It > seems > > like the config should handle controlled shutdown correctly anyway. > > > > Thanks, > > Neha > > > > > > On Thu, Mar 20, 2014 at 9:16 PM, Jun Rao <jun...@gmail.com> wrote: > > > > > We haven't been testing the ShutdownBroker command in 0.8.1 rigorously > > > since in 0.8.1, one can do the controlled shutdown through the new > config > > > "controlled.shutdown.enable". Instead of running the ShutdownBroker > > command > > > during the upgrade, you can also wait until under replicated partition > > > count drops to 0 after each restart before moving to the next one. > > > > > > Thanks, > > > > > > Jun > > > > > > > > > On Thu, Mar 20, 2014 at 3:14 PM, Ryan Berdeen <rberd...@hubspot.com> > > > wrote: > > > > > > > While upgrading from 0.8.0 to 0.8.1 in place, I observed some > > surprising > > > > behavior using kafka.admin.ShutdownBroker. At the start, there were > no > > > > underreplicated partitions. After running > > > > > > > > bin/kafka-run-class.sh kafka.admin.ShutdownBroker --broker 10 ... > > > > > > > > Partitions that had replicas on broker 10 were under-replicated: > > > > > > > > bin/kafka-topics.sh --describe --under-replicated-partitions ... > > > > Topic: analytics-activity Partition: 2 Leader: 12 Replicas: 12,10 > > > Isr: > > > > 12 > > > > Topic: analytics-activity Partition: 6 Leader: 11 Replicas: 11,10 > > > Isr: > > > > 11 > > > > Topic: analytics-activity Partition: 14 Leader: 14 Replicas: 14,10 > > > Isr: > > > > 14 > > > > ... > > > > > > > > While restarting the broker process, many produce requests failed > with > > > > kafka.common.UnknownTopicOrPartitionException. > > > > > > > > After each broker restart, I used the preferred leader election tool > > for > > > > all topics. Now, after finishing all of the broker restarts, the > > cluster > > > > seems to be stuck in leader election. Running the tool fails with > > > > "kafka.admin.AdminOperationException: Preferred replica leader > election > > > > currently in progress..." > > > > > > > > Are any of these known issues? Is there a safer way to shutdown and > > > restart > > > > brokers that does not cause producer failures and under-replicated > > > > partitions? > > > > > > > > > >
