Re: Get Latest Offset for Specific Topic for All Partition

2015-02-10 Thread Gwen Shapira
Yes, its the size of the log in bytes. We can improve on documentation a bit :) Metric is defined in Log.scala: newGauge("Size", new Gauge[Long] { def value = size }, tags) so the value is "size", and: /** * The size of the log in bytes */ def size: Long = logSegmen

Re: Get Latest Offset for Specific Topic for All Partition

2015-02-09 Thread Bhavesh Mistry
Hi Gwen, This JMX stats is good for calculate injection rate per partition. I do not have to depend on ZK to figuring out who is leader what is latest offset. One quick question, what is Size # ? is it # of bytes particular partition has on disk ? Unfortunately, MBean description is very l

Re: Get Latest Offset for Specific Topic for All Partition

2015-02-05 Thread Gwen Shapira
You can use the metrics Kafka publishes. I think the relevant metrics are: Log.LogEndOffset Log.LogStartOffset Log.size Gwen On Thu, Feb 5, 2015 at 11:54 AM, Bhavesh Mistry wrote: > HI All, > > I just need to get the latest offset # for topic (not for consumer group). > Which API to get this i

Re: Get Latest Offset for Specific Topic for All Partition

2015-02-05 Thread Joel Koshy
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowdoIaccuratelygetoffsetsofmessagesforacertaintimestampusingOffsetRequest? However, you will need to issue a TopicMetadataRequest first to discover the leaders for all the partitions and then issue the offset request. On Thu, Feb 05, 2015