Thanks for answer Otis. The producer that i use (Logstash) does not track message sizes.
I already loaded all the metrics from JMX into my monitoring system. I just need to confirm that "record" is equivalent to an individual log message. On Tue, Jun 14, 2016 at 1:27 PM, Otis Gospodnetić < otis.gospodne...@gmail.com> wrote: > Hi, > > Do you control the producers? If so, couldn't you measure the message > sizes there? > Alternatively, you can use something like SPM for Kafka or other Kafka > monitoring tools that expose relevant metrics. > > For example, I think you can compute avg msg size based on metrics shown in > these charts: > https://apps.sematext.com/spm-reports/s/T1aD50s1Xp > https://apps.sematext.com/spm-reports/s/x5WcTA470A > > Otis > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > On Mon, Jun 13, 2016 at 4:43 PM, allen chan <allen.michael.c...@gmail.com> > wrote: > > > In JMX for Kafka producer there are metrics for both request, record, and > > batch size Max + Avg. > > > > What is the difference between these concepts? > > > > In the logging use case: I assume record is the single log line, batch is > > multiple log lines together and request is the batch wrapped with the > > metadata to be sent. > > > > Can someone confirm or correct this assumption? > > I need to be able to see how big a message is so i can properly tune this > > setting: message.max.bytes. The confusion i have is do i use the request > or > > the record metric. > > > > > > Thanks, > > -- > > Allen Michael Chan > > > -- Allen Michael Chan
