Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-14 Thread Rajiv Kurian
Thanks Jason. I'll try to upgrade and see if it helps. On Mon, Mar 14, 2016 at 12:04 PM, Jason Gustafson wrote: > I think this is the one: https://issues.apache.org/jira/browse/KAFKA-2978. > > -Jason > > On Mon, Mar 14, 2016 at 11:54 AM, Rajiv Kurian wrote: > > > @Jason, can you please point me

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-14 Thread Jason Gustafson
I think this is the one: https://issues.apache.org/jira/browse/KAFKA-2978. -Jason On Mon, Mar 14, 2016 at 11:54 AM, Rajiv Kurian wrote: > @Jason, can you please point me to the bug that you were talking about in > 0.9.0.0? > > On Mon, Mar 14, 2016 at 11:36 AM, Rajiv Kurian wrote: > > > No I ha

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-14 Thread Rajiv Kurian
@Jason, can you please point me to the bug that you were talking about in 0.9.0.0? On Mon, Mar 14, 2016 at 11:36 AM, Rajiv Kurian wrote: > No I haven't. It's still running the 0.9.0 client. I'll try upgrading if > it sounds like an old bug. > > On Mon, Mar 14, 2016 at 11:24 AM, Jason Gustafson

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-14 Thread Ismael Juma
Please upgrade indeed, 0.9.0.1 includes a number of important fixes. Ismael On 14 Mar 2016 18:36, "Rajiv Kurian" wrote: > No I haven't. It's still running the 0.9.0 client. I'll try upgrading if it > sounds like an old bug. > > On Mon, Mar 14, 2016 at 11:24 AM, Jason Gustafson > wrote: > > > He

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-14 Thread Rajiv Kurian
No I haven't. It's still running the 0.9.0 client. I'll try upgrading if it sounds like an old bug. On Mon, Mar 14, 2016 at 11:24 AM, Jason Gustafson wrote: > Hey Rajiv, > > That sounds suspiciously like one of the bugs from 0.9.0.0. Have you > updated kafka-clients to 0.9.0.1? > > -Jason > > On

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-14 Thread Jason Gustafson
Hey Rajiv, That sounds suspiciously like one of the bugs from 0.9.0.0. Have you updated kafka-clients to 0.9.0.1? -Jason On Mon, Mar 14, 2016 at 11:18 AM, Rajiv Kurian wrote: > Has any one run into similar problems. I have experienced the same problem > again. This time when I use kafka-consum

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-14 Thread Rajiv Kurian
Has any one run into similar problems. I have experienced the same problem again. This time when I use kafka-consumer-groups.sh tool it says that my consumer group is either missing or rebalancing. But when I use the --list method it shows up on the list. So my guess is it is rebalancing some how.

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-09 Thread Rajiv Kurian
Thanks! That worked. So I see that for the host that I am not getting messages for has a massive lag for the 0th partition (the only one I send messages on). The other 19 groups are all caught up which explains why they have no issues. The lag is just increasing with time which confirms my suspicio

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-09 Thread Manikumar Reddy
We need to pass "--new-consumer" property to kafka-consumer-groups.sh command to use new consumer. sh kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list --new-consumer On Thu, Mar 10, 2016 at 12:02 PM, Rajiv Kurian wrote: > Hi Guozhang, > > I tried using the kafka-consumer-gro

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-09 Thread Rajiv Kurian
Hi Guozhang, I tried using the kafka-consumer-groups.sh --list command and it says I have no consumer groups set up at all. Yet I am receiving data on 19 out of 20 consumer processes (each with their own topic and consumer group). Here is my full kafka config as printed when my process started up

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-09 Thread Guozhang Wang
Rajiv, In the new Java consumer you used, the ZK dependency has been removed and hence you wont see any data from ZK path. To check the group metadata you can use the ConsumerGroupCommand, wrapped in bin/kafka-consumer-groups.sh. Guozhang On Wed, Mar 9, 2016 at 5:48 PM, Rajiv Kurian wrote: >

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-09 Thread Rajiv Kurian
Don't think I made my questions clear: On Kafka 0.9.0.1 broker and 0.9 consumer how do I tell what my consumer-groups are? Can I still get this information in ZK? I don't see anything in the consumers folder which is alarming to me. This is especially alarming because I do see that 8 partitions ar

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

2016-03-09 Thread Rajiv Kurian
Also forgot to mention that when I do consume with the console consumer I do see data coming through. On Wed, Mar 9, 2016 at 3:44 PM, Rajiv Kurian wrote: > I am running the 0.9.0.1 broker with the 0.9 consumer. I am using the > subscribe feature on the consumer to subscribe to a topic with 8 par