“We use the default log retention of 7 *days*" :)* On Wed, May 25, 2016 at 12:34 PM, Andrew Otto <o...@wikimedia.org> wrote:
> Hiya, > > We’ve recently upgraded to 0.9. In 0.8, when we restarted a broker, data > log file mtimes were not changed. In 0.9, any data log file that was on > disk before the broker has it’s mtime modified to the time of the broker > restart. > > This causes problems with log retention, as all the files then look like > they contain recent data to kafka. We use the default log retention of 7 > weeks, but if all the files are touched at the same time, this can cause us > to retain up to 2 weeks of log data, which can fill up our disks. > > We saw this during our initial upgrade, but I had just thought it had > something to do with the change of inter.broker.protocol.version, and > assumed it wouldn’t happen again. We just did our first broker restart > after the upgrade, and we are seeing this again. We worked around this > during our upgrade by temporarily setting a high volume topic’s retention > very low, causing brokers to purge more recent data. This allowed us to > avoid filling up our disks, but we shouldn’t have to do this every time we > bounce brokers. > > Has anyone else noticed this? > > -Ao >
