Re: Problem of FetchRequest API

2013-11-20 Thread 小宇
My apologies, I made mistake, the order in which Kafka server handle requests has no problem. Thanks. 2013/11/21 Jun Rao > Does the built-in java client have the same problem? > > Thanks, > > Jun > > > On Wed, Nov 20, 2013 at 7:00 PM, 小宇 wrote: > > > Yes, it is reproducible. I have reported it

Re: Problem of FetchRequest API

2013-11-20 Thread Jun Rao
Does the built-in java client have the same problem? Thanks, Jun On Wed, Nov 20, 2013 at 7:00 PM, 小宇 wrote: > Yes, it is reproducible. I have reported it in Jira: > https://issues.apache.org/jira/browse/KAFKA-1137 . BTW, we meet this > problem when wirte a Node.js Client of Kafka0.8 , > https

Re: Problem of FetchRequest API

2013-11-20 Thread 小宇
Yes, it is reproducible. I have reported it in Jira: https://issues.apache.org/jira/browse/KAFKA-1137 . BTW, we meet this problem when wirte a Node.js Client of Kafka0.8 , https://github.com/SOHU-Co/kafka-node. Thanks. 2013/11/20 Jun Rao > That sounds like a bug. Is that reproducible? If so, co

Re: Kafka List Topic Command O/P and Jconsole o/p differs

2013-11-20 Thread Joel Koshy
Did you send any messages to those topics? I think we register those mbeans only after messages have been received. On Wed, Nov 20, 2013 at 03:20:25PM +0530, Monika Garg wrote: > Hi, > I am getting a very surprising case : > > I am having a 2 nodes Kafka cluster and created some topics. > > When

Re: List of topics with JMX?

2013-11-20 Thread Hanish Bansal
Hi Andrew, You are right. It is count of messages in all topics that have been received by the broker since the broker process started. It is just counter value that get incremented each time a new message is received by broker. The value is reset to 0 when broker is restarted. Also the message co

Re: Problem of FetchRequest API

2013-11-20 Thread Jun Rao
That sounds like a bug. Is that reproducible? If so, could you file a jira? Thanks, Jun On Wed, Nov 20, 2013 at 12:05 AM, 小宇 wrote: > Hi, > I am now using kafka0.8, i have problem with FetchRequest API ( > > https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#A

Re: processing a batch of messages in a "transaction"

2013-11-20 Thread Imran Rashid
perfect, thank you! On Wed, Nov 20, 2013 at 8:44 AM, Neha Narkhede wrote: > You can turn off automatic offset commit (auto.commit.enable=false) and use > the commitOffsets() API. Note that this API will commit offsets for all > partitions owned by the consumer. > > Thanks, > Neha > On Nov 20, 201

Re: processing a batch of messages in a "transaction"

2013-11-20 Thread Neha Narkhede
You can turn off automatic offset commit (auto.commit.enable=false) and use the commitOffsets() API. Note that this API will commit offsets for all partitions owned by the consumer. Thanks, Neha On Nov 20, 2013 6:39 AM, "Imran Rashid" wrote: > Hi, > > I have an application which reads messages f

Re: Kafka List Topic Command O/P and Jconsole o/p differs

2013-11-20 Thread Neha Narkhede
Can you send around steps to reproduce this? Thanks, Neha On Nov 20, 2013 1:50 AM, "Monika Garg" wrote: > Hi, > I am getting a very surprising case : > > I am having a 2 nodes Kafka cluster and created some topics. > > When I am running the command : > > *bin/kafka-list-topic.sh --zookeeper 192.

processing a batch of messages in a "transaction"

2013-11-20 Thread Imran Rashid
Hi, I have an application which reads messages from a kafka queue, builds up a batch of messages, and then performs some action on that batch. So far I have just used the ConsumerGroup api. However, I realized there is a potential problem -- my app may die sometime in the middle of the batch, and

Re: Leader = -1 is coming when broker goes down

2013-11-20 Thread Neha Narkhede
If all replicas for a partition go offline, then leader will be set to -1. Thanks, Neha On Nov 20, 2013 1:45 AM, "Monika Garg" wrote: > Hi Neha, > > In context of the same mail thread that you have already answered,I just > want to clear one more doubt: > > If I am making a cluster with two brok

Re: How to get monitoring stats

2013-11-20 Thread Andrew Otto
For Kafka 0.8, you could use http://www.jmxtrans.org/ and a variation of this json file: https://github.com/wikimedia/puppet-kafka/blob/master/kafka-jmxtrans.json.md Just change the ganglia output writers to graphite ones. On Nov 19, 2013, at 8:25 PM, Benjamin Black wrote: > https://www.goo

Kafka List Topic Command O/P and Jconsole o/p differs

2013-11-20 Thread Monika Garg
Hi, I am getting a very surprising case : I am having a 2 nodes Kafka cluster and created some topics. When I am running the command : *bin/kafka-list-topic.sh --zookeeper 192.168.145.119:2181 *, I am getting the below output: topic: sample-tickpartition: 0

Re: Leader = -1 is coming when broker goes down

2013-11-20 Thread Monika Garg
Hi Neha, In context of the same mail thread that you have already answered,I just want to clear one more doubt: If I am making a cluster with two brokers and a topic with 3 partitions and two replicas,then can leader = -1 come here in any case? On Tue, Oct 22, 2013 at 10:55 PM, Monika Garg wr

Problem of FetchRequest API

2013-11-20 Thread 小宇
Hi, I am now using kafka0.8, i have problem with FetchRequest API ( https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-FetchRequest), the detail of the problem as follow: I send two FetchRequest [topic, partition, maxWaitTime] through 1 sock