Re: How to monitor Kafka offset

2015-07-09 Thread Rahul Jain
Burrow works only if you are storing the offsets in kafka topic, not zookeeper. You can also take a look at Kafka web console ( it has a memory leak bug but a patch is available ). On 10 Jul 2015 09:34, "Jiangjie Qin" wrote: > You can take a look at Burrow. We use it in LinkedIn. > > Thanks, > >

Re: Got conflicted ephemeral node exception for several hours

2015-07-09 Thread tao xiao
Bump the thread. Any help would be appreciated. On Wed, 8 Jul 2015 at 20:09 tao xiao wrote: > Additional info > Kafka version: 0.8.2.1 > zookeeper: 3.4.6 > > On Wed, 8 Jul 2015 at 20:07 tao xiao wrote: > >> Hi team, >> >> I have 10 high level consumers connecting to Kafka and one of them kept >

Re: How to monitor Kafka offset

2015-07-09 Thread Jiangjie Qin
You can take a look at Burrow. We use it in LinkedIn. Thanks, Jiangjie (Becket) Qin On 7/9/15, 8:53 PM, "Anandh Kumar" wrote: >Hi > >We have any monitor tool which monitor kafka offset? > >I need some opensource admin tool for kafka. > >Please guide me. > >Regards, >-Anandh Kumar

How to monitor Kafka offset

2015-07-09 Thread Anandh Kumar
Hi We have any monitor tool which monitor kafka offset? I need some opensource admin tool for kafka. Please guide me. Regards, -Anandh Kumar

Re: ISR not a replica

2015-07-09 Thread Krishna Kumar
Well, 3 (the new node) was shut down, so there were no messages there. “1" was the leader and we saw the messages on “0” and “2”. We managed to resolve this new problem to an extent by shutting down “1". We were worried because “1” was the only replica in the ISR. But once it went down, “0” and “2

Re: ISR not a replica

2015-07-09 Thread Guozhang Wang
Krish, Does broker 0 and 3 have the similar warn log entries as broker 2 for stale controller epochs? Guozhang On Thu, Jul 9, 2015 at 2:07 PM, Krishna Kumar wrote: > So we tried taking that node down. But that didn¹t fix the issue, so we > restarted the other nodes. > > This seems to have lead

Re: Launch Kafka/Zookeeper servers without using command line

2015-07-09 Thread Jeff Gong
Hey Gwen, Thanks for your response. The example was very helpful and exactly what I was looking for! I went ahead and used that as an example and while most things are working as I wanted, there are a few issues I'd like to follow up on: 1. If I launch a Zookeeper/Kafka server from code instead o

Re: kafka consumer group API

2015-07-09 Thread noah
Hi! I did something similar. You can use the high level consumer but turn off auto commit and commit only what you are done with. Here's the code I used: https://github.com/iamnoah/kakfa-offsets-test On Thu, Jul 9, 2015 at 4:53 PM Shashank Singh wrote: > Hi Team > > I was going over the documen

kafka consumer group API

2015-07-09 Thread Shashank Singh
Hi Team I was going over the documentation and links related to consumer API provided in 0.8.2.1 release I got to know of major improvement in consumer API in 0.8.3 release. May I know when we are planning to have this release. One more help - I was following up the example given on https://cwiki

Re: ISR not a replica

2015-07-09 Thread Krishna Kumar
So we tried taking that node down. But that didn¹t fix the issue, so we restarted the other nodes. This seems to have lead to 2 of other replicas dropping out of the ISIR for *all* topics. Topic: topic2 Partition: 0 Leader: 1 Replicas: 1,0,2 Isr: 1 Topic: topic2 Partition: 1

Re: ISR not a replica

2015-07-09 Thread Krishna Kumar
Thanks Guozhang We did do the partition-assignment, but against another topic, and that went well. But this happened for this topic without doing anything. Regards Krish On 7/9/15, 3:56 PM, "Guozhang Wang" wrote: >Krishna, > >Did you run any admin tools after adding the node (I assume it is n

Re: ISR not a replica

2015-07-09 Thread Guozhang Wang
Krishna, Did you run any admin tools after adding the node (I assume it is node 3), like partition-assignment? It is shown as the only one in ISR list but not in the replica list, which seems that the partition migration process was not completed. You can verify if this is the case by checking yo

ISR not a replica

2015-07-09 Thread Krishna Kumar
Hi We added a Kafka node and it suddenly became the leader and the sole replica for some partitions, but it is not in the ISR Any idea how we might be able to fix this? We are on Kafka 0.8.2 Topic: topic1 Partition: 0 Leader: 2 Replicas: 2,1,0 Isr: 2,0,1 Topic: topic1 Partitio

Re: Leap Second Troubles

2015-07-09 Thread Todd Palino
Did you hit the problems in the Kafka brokers and consumers during the Zookeeper problem, or after you had already cleared it? For us, we decided to skip the leap second problem (even though we're supposedly on a version that doesn't have that bug) by shutting down ntpd everywhere and then allowin

Leap Second Troubles

2015-07-09 Thread Christofer Hedbrandh
Hi Kafka users, ZooKeeper in our staging environment was running on a very old ubuntu version, that was exposed to the "leap second causes spuriously high CPU usage" bug. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1020285 As a result, when the leap second arrived, the ZooKeeper CPU usa

Re: Kafka as an event store for Event Sourcing

2015-07-09 Thread Daniel Schierbeck
Ben, are you still interested in working on this? On Mon, Jun 15, 2015 at 9:49 AM Daniel Schierbeck < daniel.schierb...@gmail.com> wrote: > I like to refer to it as "conditional write" or "conditional request", > semantically similar to HTTP's If-Match header. > > Ben: I'm adding a comment about

Re: How can I unsubscribe from this mailing list

2015-07-09 Thread tao xiao
You can follow the instruction listed here http://kafka.apache.org/contact.html In short send an email to users-unsubscr...@kafka.apache.org On Thu, 9 Jul 2015 at 18:52 Monika Garg wrote: > Still I am not able to unsubscribe and still getting mails. > > > Please help me out as It is filling my

Re: How can I unsubscribe from this mailing list

2015-07-09 Thread Monika Garg
Still I am not able to unsubscribe and still getting mails. Please help me out as It is filling my inbox. On Fri, Jul 3, 2015 at 11:57 AM, Monika Garg wrote: > Thanks. > > I will try these to unsubscribe. > > On Thu, Jun 25, 2015 at 12:59 PM, Stevo Slavić wrote: > >> I believe it's: users-uns