Re: Consuming a snapshot from log compacted topic

2015-04-14 Thread Will Funnell
Hi, Any update on the above patch? Hoping you might be able to review it soon. Thanks. On 23 February 2015 at 21:21, Will Funnell wrote: > Hey guys, > > I created a patch based on your feedback. > > Let me know what you think. > > https://issues.apache.org/jira/browse/KAFKA-1977 > > On 20 F

Re: Fetch Request Purgatory and Mirrormaker

2015-04-14 Thread Evan Huus
Any ideas on this? It's still occurring... Is there a separate mailing list or project for mirrormaker that I could ask? Thanks, Evan On Thu, Apr 9, 2015 at 4:36 PM, Evan Huus wrote: > Hey Folks, we're running into an odd issue with mirrormaker and the fetch > request purgatory on the brokers.

Automatic recreation of the topic after the topic was deleted

2015-04-14 Thread Pavel Bžoch
Hi all, I have a question about automatic re-/creation of the topic. I started my broker with property "auto.create.topics.enable " set to true. Then I tried to produce some messages to non-existing topic. From the broker log, I saw that the topic was created. Then I deleted this topic using AdminU

Re: Automatic recreation of the topic after the topic was deleted

2015-04-14 Thread Gwen Shapira
Were there any errors on broker logs when you sent the new batch of messages? On Tue, Apr 14, 2015 at 7:41 AM, Pavel Bžoch wrote: > Hi all, > I have a question about automatic re-/creation of the topic. I started my > broker with property "auto.create.topics.enable " set to true. Then I tried > t

Re: serveral questions about auto.offset.reset

2015-04-14 Thread James Cheng
"What to do when there is no initial offset in ZooKeeper or if an offset is out of range" I personally find the name "auto.offset.reset" to be somewhat confusing. That's mostly because I only knew of it as the "no initial offset" setting. An alternate name could be "auto.offset.initial", to ha

Re: Consumer offsets in offsets topic 0.8.2

2015-04-14 Thread 4mayank
Thanks guys. Based on the responses it looks like I need to have a tool of my own to read offsets. On Mon, Apr 13, 2015 at 3:07 PM, Jiangjie Qin wrote: > Yeah, the current ConsumerOffsetChecker has this issue (maybe bug also) if > the offset storage is Kafka and no offset has been committed. It

Design questions related to kafka

2015-04-14 Thread Sivananda Reddy
Hi, # I looked the documents of kafka and I see that there is no way a consume instance can read specific messages from partition. # I have an use case where I need to spawn a topic(single partition) for each user, so there would be 10k online users at a time, there would be

Re: Fetch Request Purgatory and Mirrormaker

2015-04-14 Thread Jiangjie Qin
Hey Evan, Is this issue only observed when mirror maker is consuming? It looks that for Cluster A you have some other consumers. Do you mean if you stop mirror maker the problem goes away? Jiangjie (Becket) Qin On 4/14/15, 6:55 AM, "Evan Huus" wrote: >Any ideas on this? It's still occurring...

Re: Fetch Request Purgatory and Mirrormaker

2015-04-14 Thread Evan Huus
On Tue, Apr 14, 2015 at 8:31 PM, Jiangjie Qin wrote: > Hey Evan, > > Is this issue only observed when mirror maker is consuming? It looks that > for Cluster A you have some other consumers. > Do you mean if you stop mirror maker the problem goes away? > Yes, exactly. The setup is A -> Mirrormake