On Tuesday 13 December 2016 03:29 PM, Ismael Juma wrote:
The log config settings for the controller and state change logger have
been that way since they were introduced. They're generally useful when
investigating issues with the controller. Looks like this is too noisy in
some scenarios though. It may be worth filing a JIRA with specifics of your
use case to see if something can be done to improve that.
For now, we decided to use a custom log4j config. I'll get back to this
and check if there are specific log messages that cause this or if it's
just regular logs at TRACE level which contribute to this, before filing
a JIRA.
-Jaikiran
Ismael
On Tue, Dec 13, 2016 at 9:32 AM, Jaikiran Pai <jai.forums2...@gmail.com>
wrote:
We happened to run into a disk space usage issue with Kafka 0.10.0.1 (the
version we are using) on one of our production setups this morning. Turns
out (log4j) logging from Kafka ended up using 81G and more of disk space.
Looking at the files, I see the controller.log itself is 30G and more (for
a single day). Looking at the default log4j.properties that's shipped in
Kafka, it uses the DailyRollingFileAppender which is one of the things that
contributes to this issue. I see that there's already a patch and JIRA to
fix this https://issues.apache.org/jira/browse/KAFKA-2394. It's been
marked for 0.11 because there wasn't a clear decision when to ship it.
Given that we have been going through 0.10.x releases these days and the
0.11 release looking a bit away, is there any chance, this specific JIRA
can make it to 0.10.x? I personally don't see any compatibility issues that
it will introduce when it comes to *functionality/features* of Kafka
itself, so I am not sure if it's that big a change to wait all the way till
0.11. Furthermore, since the default shipped setting can cause issues like
the one I noted, I think it probably would be fine to include it in one of
the 0.10.x releases. Of course, we ourselves can setup the logging config
on our setup to use a size based rolling file config instead of the one
shipped by default, but if this is something that can make it to 0.10.x, we
would like to avoid doing this customization ourselves.
That's one part of the issue. The other is, I see this in the default
shipped log4j.properties:
log4j.logger.kafka.controller=*TRACE,* controllerAppender
log4j.additivity.kafka.controller=false
log4j.logger.state.change.logger=*TRACE*, stateChangeAppender
log4j.additivity.state.change.logger=false
Is it intentional to have this at TRACE level for the default shipped
config instead of having something like INFO or maybe DEBUG?
-Jaikiran
