Re: Kafka Queue Depth Metrics

2014-10-05 Thread Jun Rao
This is a jmx on the consumer side. The server side metric is for the internal replication consumer and is only available if you have a replication factor more than 1. Thanks, Jun On Fri, Oct 3, 2014 at 4:28 AM, Shah, Devang1 wrote: > I tried hooking up jconsole to Kafka server but could not s

Re: auto topic creation not working for attempts to consume non-existing topic

2014-10-05 Thread Jun Rao
Yes, the docs can be improved. Could you file a jira? For the 2nd issue, the new java producer handles this better. Thanks, jun On Fri, Oct 3, 2014 at 1:31 AM, Stevo Slavić wrote: > OK, thanks, > > Do you agree then that the docs for auto topic creation configuration > parameter are misleadin

Re: programmatically get number of items in topic/partition

2014-10-05 Thread Joe Stein
Take a look into how Offset Requests are made and handled. You could do that to get the low and high watermarks for the stream. With the high and low watermark just subtract them to get total. This will be in the consumer api part of whatever language client your using. The specs for this are htt

Re: programmatically get number of items in topic/partition

2014-10-05 Thread Shlomi Hazan
actually this tool is not a 100% match to what I need, since it can only provide information on topics that have comsumers: Is there also another equivalent tool/method of querying topics that have no consumers ? in this case this tool will not help as it requires a group id as a mandatory paramete

Re: programmatically get number of items in topic/partition

2014-10-05 Thread Shlomi Hazan
Bingo. 10x!! On Wed, Oct 1, 2014 at 6:41 PM, chetan conikee wrote: > The other method is via command line > > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group > *groupName* > --zkconnect *zkServer:2181* > > Refer : > > https://cwiki.apache.org/confluence/display/KAFKA/System+Tool