On server side, we use INFO for everything. Log4j setting can be a temporary hack, but we would like to keep INFO logging as the default.
I think those two logging lines can be just downgraded into DEBUG loggings, moving start offset is not that eventful to be logged as INFO. On Fri, Jul 6, 2018 at 4:08 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hello Henry, > > What's your server-side log4j settings? Could you use WARN on these two > classes: kafka.server.epoch.LeaderEpochFileCache and kafka.log.Log. > > > > Guozhang > > > On Fri, Jul 6, 2018 at 3:08 PM, Henry Cai <h...@pinterest.com.invalid> > wrote: > > > @guozhang > > > > After we moved to kafka-1.1.0 for our Kafka streams application, our > broker > > logs are polluted with loggings such as: > > > > [2018-07-06 21:59:26,170] INFO Cleared earliest 0 entries from epoch > cache > > based on passed offset 301483601 leaving 1 in EpochFile for partition > > inflight_spend_unified_staging-single_spend_agg_ > > window_AD_GROUP-repartition-26 > > (kafka.server.epoch.LeaderEpochFileCache) > > > > [2018-07-06 21:59:26,170] INFO [Log > > partition=inflight_spend_unified_staging-single_spend_ > agg_window_AD_GROUP- > > repartition-1, > > dir=/mnt/kafka] Incrementing log start offset to 240548684 > (kafka.log.Log) > > > > > > Thousands of them keeping rolling in the broker logs which makes server > > side log unusable. > > > > > > Looks like this is triggered by DELETE_RECORDS requests from > StreamsThread > > from 'KAFKA-6150: Make Repartition Topics Transient' > > > > > > Can you suppress these two INFO loggings on the server side if they are > > triggered by AdminClient.deleteRecords() > > > > > > We have thousands of partitions per broker, those deletes were happening > > too frequent. > > > > > > -- > -- Guozhang >
