Re: kafka.SocketServerStats write rate metrics seem wrong

2014-08-06 Thread Jun Rao
TotalBytesWritten is probably for sending the responses. You probably should look at TotalBytesRead for incoming data rate. Thanks, Jun On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins wrote: > Hi. > I'm seeing some odd numbers from kafka.SocketServerStats. > > Ideally, I'd like to have stats br

Re: kafka.SocketServerStats write rate metrics seem wrong

2014-08-05 Thread Neil Harkins
No compression is configured, the spool files can be /usr/bin/strings'd to show the msgs. On Tue, Aug 5, 2014 at 2:13 PM, Guozhang Wang wrote: > Hello Neil, > > Did you use any compression on your producers? > > Guozhang > > > On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins wrote: >> >> Hi. >> I'

Re: kafka.SocketServerStats write rate metrics seem wrong

2014-08-05 Thread Guozhang Wang
Hello Neil, Did you use any compression on your producers? Guozhang On Tue, Aug 5, 2014 at 12:23 PM, Neil Harkins wrote: > Hi. > I'm seeing some odd numbers from kafka.SocketServerStats. > > Ideally, I'd like to have stats broken down per-topic, e.g. > what's our most written/read topics? For

kafka.SocketServerStats write rate metrics seem wrong

2014-08-05 Thread Neil Harkins
Hi. I'm seeing some odd numbers from kafka.SocketServerStats. Ideally, I'd like to have stats broken down per-topic, e.g. what's our most written/read topics? For write rates, I've got a separate process iterating topics every minute, doing this: (head_offset_now - head_offset_last) / (time_now -