[ https://issues.apache.org/jira/browse/KAFKA-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13956002#comment-13956002 ]
Joel Koshy commented on KAFKA-1355: ----------------------------------- BTW, here are the configs and steps I used for this, KAFKA-1342 and KAFKA-1350: Four brokers, 100 topics, eight partitions each. Log4j and server configs by broker: https://gist.github.com/anonymous/9906088 https://gist.github.com/anonymous/9906092 https://gist.github.com/anonymous/9906096 https://gist.github.com/anonymous/9906102 https://gist.github.com/anonymous/9906144 https://gist.github.com/anonymous/9906148 https://gist.github.com/anonymous/9906153 https://gist.github.com/anonymous/9906157 Producer performance: https://gist.github.com/anonymous/9906163 (At the end, just grep in the controller's request log and extract local time) grep ControlledShutdownRequest logs/kafka-request* > Reduce/optimize update metadata requests sent during leader election > -------------------------------------------------------------------- > > Key: KAFKA-1355 > URL: https://issues.apache.org/jira/browse/KAFKA-1355 > Project: Kafka > Issue Type: Bug > Affects Versions: 0.8.1 > Reporter: Joel Koshy > Priority: Critical > Fix For: 0.8.1.1 > > > This is part of the investigation into slow shutdowns in 0.8.1. While > logging contributes to bulk of the regression, this one also adds > quite a bit of overhead: > In addLeaderAndIsrRequest (called for every partition that is led by the > broker being shut down) we also add an UpdateMetadataRequest - each call to > addUpdateMetadataRequests does two traversals over _all_ (global) > partitions. I think it should be straightforward to optimize this a bit. > Marking as critical, since it is not as big an overhead as the logging. -- This message was sent by Atlassian JIRA (v6.2#6252)
