[
https://issues.apache.org/jira/browse/KAFKA-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13818405#comment-13818405
]
Jason Rosenberg commented on KAFKA-1122:
----------------------------------------
Well, I just think log files should be something you can check on, to see
things look normal, and then you can easily to searches over (e.g. in splunk,
or something similar). So, having single lines that are 50K long, is simply
not useful. You mention grep, how would grepping for a string in a giant log
line be helpful?
If say I want to know everything that happened for a particular topic or
partition, it's not helpful if it appears in a logline with also, 700 other
topics.
I think breaking things up, so they are coherent for doing useful searches, and
also, so that you can visually inspect a log file, and see that there are no
unexpected exceptions, is useful. You can't do that easily when some lines are
too long.
If I'm restarting the server, I will generally tail the log file for a while,
to make sure it looks good.
> Kafka can log giant log lines
> -----------------------------
>
> Key: KAFKA-1122
> URL: https://issues.apache.org/jira/browse/KAFKA-1122
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jason Rosenberg
> Priority: Minor
>
> There are a number of log lines that the kafka server, and high-level
> consumer, can log, that can end up becoming a giant log line. This can be
> cumbersome to deal with in a log file.
> This happens in my case as I have have a large number of topics (on the order
> of 500-700 topics). Typically, these giant log lines will say something
> separately about every topic on the broker. An example:
> 2013-11-04 23:28:11,148 INFO [kafka-request-handler-0] server.ReplicaManager
> - [Replica Manager on Broker 10]: Handling LeaderAndIsr request
> Name:LeaderAndIsrRequest;Version:0;Controller:11;ControllerEpoch:220;CorrelationId:5;ClientId:id_11-host_null-port_27330;PartitionState:(mytopic,0)
> ->
> (LeaderAndIsrInfo:(Leader:11,ISR:11,LeaderEpoch:43,ControllerEpoch:219),ReplicationFactor:2),.....
> Imagine that line going on with a separate entry for 700 topics. There are
> many other examples of this phenomenon in the server, and high-level consumer.
> I'd think these log lines could be separated into a single line per topic.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
