Jiang,
Thanks for the KIP. I think it make sense. I agree with Sophie that the
KIP writeup should be improved a little bit, with regard to public API
(ie, config) which are changed.
The only other idea I had to avoid this issue would be some internal
change: we would introduce a new logger class allowing you to disable
logging for this specific logger. However, not sure how we could make
this a public contract you could rely on? -- Also, given Sophie's
comment about potential other logs we might want to configure, it seems
like a good idea to add this config.
@Sophie: what other logs did you have in mind?
One more nit: rejected alternatives lists `log.summary.interval.ms` as
rejected -- seems this needs to be removed?
-Matthias
On 5/29/24 12:56 AM, Sophie Blee-Goldman wrote:
Sure, as I said I'm supportive of this KIP. Just wanted to mention how the
issue could be mitigated in the meantime since the description made it
sound like you were suffering from excessive logs right now. Apologies if I
misinterpreted that.
I do think it would be nice to have a general setting for log intervals in
Streams. There are some other places where a regular summary log might be
nice. The config name you proposed is generic enough that we could reuse it
for other areas where we'd like to log summaries, so this seems like a good
config to introduce
My only question/request is that the KIP doesn't mention where this config
is being added. I assume from the context and Motivation section that
you're proposing to add this to StreamsConfig, which makes sense to me. But
please update the KIP to say this somewhere.
Otherwise the KIP LGTM. Anyone else have thoughts on this?
On Thu, May 23, 2024 at 12:19 AM jiang dou <dujian0...@gmail.com> wrote:
Thank you for your reply，
I do not recommend agreeing set log level is WARN， because INFO level logs
should be useful
Sophie Blee-Goldman <sop...@responsive.dev> 于2024年5月23日周四 04:30写道：
Thanks for the KIP!
I'm not against adding this as a config for this per se, but if this is
causing you trouble right now you should be able to disable it via log4j
configuration so you don't need to wait for a fix in Kafka Streams
itself.
Putting something like this in your log4j will shut off the offending
log:
log4j.logger.org.apache.kafka.streams.processor.internals.StreamThread=WARN
On Wed, May 22, 2024 at 6:46 AM jiang dou <dujian0...@gmail.com> wrote:
Hi
I would like to propose a change in the kafka-stream summary log。
Now the summary of stream-tread is record every two minutes, and not
support close or update intervals.
When the kafka is running， this is absolutely unnecessary and even
harmful
since it fills the logs and thus storage space with unwanted and
useless
data.
I propose adding a configuration to control the output interval or
disable
it
KIP：
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1049%3A+Add+config+log.summary.interval.ms+to+Kafka+Streams
