Re: Topic doesn't exist exception

There's some good advice here: https://kafka.apache.org/081/ops.html And you may enjoy this post too: http://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/ On Fri, Oct 17, 2014 at 5:52 PM, Mohit Anchlia wrote: > Thanks for the info. I see there are tons of parameters but is the

Re: Topic doesn't exist exception

Thanks for the info. I see there are tons of parameters but is there a place that lists some important performance specific parameters? On Fri, Oct 17, 2014 at 2:43 PM, Gwen Shapira wrote: > If I understand correctly (and I'll be happy if someone who knows more > will jump in and correct me): > >

Re: Topic doesn't exist exception

If I understand correctly (and I'll be happy if someone who knows more will jump in and correct me): The Sync/Async part is not between the producer and the broker. Its between you and the producer. The Sync producer takes your message and immediately contacts the broker, sends the message, either

Re: Topic doesn't exist exception

My understanding of sync is that producer waits on .send until Kafka receives the message. And async means it just dispatches the message without any gurantees that message is delivered. Did I get that part right? On Fri, Oct 17, 2014 at 1:28 PM, Gwen Shapira wrote: > Sorry if I'm confusing you :

Re: Topic doesn't exist exception

Sorry if I'm confusing you :) Kafka 0.8.1.1 has two producers sync and async. You are using the sync producer without waiting for acks. I hope this helps? Regardless, did you check if the partition got created? are you able to produce messages? are you able to consume them? Gwen On Fri, Oct 17,

Re: Topic doesn't exist exception

Still don't understand the difference. If it's not waiting for the ack then doesn't it make async? On Fri, Oct 17, 2014 at 12:55 PM, wrote: > Its using the sync producer without waiting for any broker to acknowledge > the write. This explains the lack of errors you are seeing. > > — > Sent from

Re: Topic doesn't exist exception

Its using the sync producer without waiting for any broker to acknowledge the write. This explains the lack of errors you are seeing. — Sent from Mailbox On Fri, Oct 17, 2014 at 3:15 PM, Mohit Anchlia wrote: > Little confused :) From one of the examples I am using property > request.required.

Re: Topic doesn't exist exception

Little confused :) From one of the examples I am using property request.required.acks=0, I thought this sets the producer to be async? On Fri, Oct 17, 2014 at 11:59 AM, Gwen Shapira wrote: > 0.8.1.1 producer is Sync by default, and you can set producer.type to > async if needed. > > On Fri, Oct 1

Re: Topic doesn't exist exception

0.8.1.1 producer is Sync by default, and you can set producer.type to async if needed. On Fri, Oct 17, 2014 at 2:57 PM, Mohit Anchlia wrote: > Thanks! How can I tell if I am using async producer? I thought all the > sends are async in nature > On Fri, Oct 17, 2014 at 11:44 AM, Gwen Shapira > wro

Re: Topic doesn't exist exception

Thanks! How can I tell if I am using async producer? I thought all the sends are async in nature On Fri, Oct 17, 2014 at 11:44 AM, Gwen Shapira wrote: > If you have "auto.create.topics.enable" set to "true" (default), > producing to a topic creates it. > > Its a bit tricky because the "send" that

Re: Topic doesn't exist exception

If you have "auto.create.topics.enable" set to "true" (default), producing to a topic creates it. Its a bit tricky because the "send" that creates the topic can fail with "leader not found" or similar issue. retrying few times will eventually succeed as the topic gets created and the leader gets e

Topic doesn't exist exception

Is Kafka supposed to throw exception if topic doesn't exist? It appears that there is no exception thrown even though no messages are delivered and there are errors logged in Kafka logs.