I noticed the same issue too with 0.9.

On Wed, 25 May 2016 at 09:49 Andrew Otto <o...@wikimedia.org> wrote:

> “We use the default log retention of 7 *days*" :)*
>
> On Wed, May 25, 2016 at 12:34 PM, Andrew Otto <o...@wikimedia.org> wrote:
>
> > Hiya,
> >
> > We’ve recently upgraded to 0.9.  In 0.8, when we restarted a broker, data
> > log file mtimes were not changed.  In 0.9, any data log file that was on
> > disk before the broker has it’s mtime modified to the time of the broker
> > restart.
> >
> > This causes problems with log retention, as all the files then look like
> > they contain recent data to kafka.  We use the default log retention of 7
> > weeks, but if all the files are touched at the same time, this can cause
> us
> > to retain up to 2 weeks of log data, which can fill up our disks.
> >
> > We saw this during our initial upgrade, but I had just thought it had
> > something to do with the change of inter.broker.protocol.version, and
> > assumed it wouldn’t happen again.  We just did our first broker restart
> > after the upgrade, and we are seeing this again.  We worked around this
> > during our upgrade by temporarily setting a high volume topic’s retention
> > very low, causing brokers to purge more recent data.  This allowed us to
> > avoid filling up our disks, but we shouldn’t have to do this every time
> we
> > bounce brokers.
> >
> > Has anyone else noticed this?
> >
> > -Ao
> >
>

Reply via email to