Hi Jie, If a broker shuts down cleanly, it leaves a .clean_shutdown file in the broker's data directory (log.dirs). On startup, a broker skips recovery of log files if it finds that file. If it doesn't, then it goes through a recovery process where the last segment for every partition is checked for consistency. If this happens, the time for broker startup is proportional to the number of partitions and the segment size on the broker.
Could you send around just the INFO level log from the broker during the long startup? I'd like to see if it was slow due to log recovery. Thanks, Neha On Tue, Mar 25, 2014 at 6:00 PM, Jie Li <jay23j...@gmail.com> wrote: > Hello, > > This is Jie from Pinterest. We did a upgrade from 0.8.0 to 0.8.1 today > with rolling restart, but found some brokers took a very long time to > restart while some just restarted immediately. > > Here are some debug logs: > > [2014-03-25 23:17:13,381] DEBUG Adding index entry 41731335 => 248915234 to > 00000000000041632481.index. (kafka.log.OffsetIndex) > [2014-03-25 23:17:13,381] DEBUG Adding index entry 41731338 => 248922262 to > 00000000000041632481.index. (kafka.log.OffsetIndex) > [2014-03-25 23:17:13,381] DEBUG Adding index entry 41731340 => 248928782 to > 00000000000041632481.index. (kafka.log.OffsetIndex) > [2014-03-25 23:17:13,381] DEBUG Adding index entry 41731343 => 248935723 to > 00000000000041632481.index. (kafka.log.OffsetIndex) > [2014-03-25 23:17:13,381] DEBUG Adding index entry 41731346 => 248942845 to > 00000000000041632481.index. (kafka.log.OffsetIndex) > [2014-03-25 23:17:13,381] DEBUG Adding index entry 41731348 => 248949551 to > 00000000000041632481.index. (kafka.log.OffsetIndex) > > Does anyone has idea? > > Thanks, > Jie >
