Matthias J. Sax created KAFKA-8251:
--------------------------------------
Summary: Consider different logging for purgeData
Key: KAFKA-8251
URL: https://issues.apache.org/jira/browse/KAFKA-8251
Project: Kafka
Issue Type: Improvement
Components: core
Reporter: Matthias J. Sax
As reported in this SO question
[https://stackoverflow.com/questions/55724259/kafka-streams-floods-kafka-logs/55738257#55738257]
each time data is purged from a topic a log message is written:
> {{[2019-04-17 09:06:16,541] INFO [Log
>partition=my-application-KSTREAM-AGGREGATE-STATE-STORE-0000000076-repartition-0,
> dir=/opt/kafka/data/logs] Incrementing log start offset to 316423
>(kafka.log.Log) [2019-04-17 09:06:16,545] INFO [Log
>partition=my-application-KSTREAM-AGGREGATE-STATE-STORE-0000000033-repartition-2,
> dir=/opt/kafka/data/logs] Incrementing log start offset to 3394
>(kafka.log.Log) }}
While this is useful for regular log truncation, it seems less useful for
explicit purge-data requests. Kafka Streams uses the purge data API on
repartition topics aggressively and this can result to verbose logging.
I don't know this part of the code, but maybe it's possible to distinguish both
cases and only log on time/size based truncation at INFO level and reduce log
level for purge-data request to TRACE?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
