I think that the attempt to write a message to a non-existent topic creates that topic (when auto.create is set to true). If it's set to false you get back error. Have you tried that?
Marko Bonaći Monitoring | Alerting | Anomaly Detection | Centralized Log Management Solr & Elasticsearch Support Sematext <http://sematext.com/> | Contact <http://sematext.com/about/contact.html> On Fri, Jan 8, 2016 at 5:51 PM, Cosmin Marginean <cosmargin...@gmail.com> wrote: > Hi Dana > > I am actually creating the topic manually from our code. This is mainly > because I couldn’t tell from the documentation when exactly is the topic > auto-created - maybe you can shed some light on that. > > The docs (http://kafka.apache.org/documentation.html) only mention > auto.create.topics.enable Enable auto creation of topic on the server > which is a bit without context. > > Either way (automated or not), it is crucial for us to know if the message > actually gets sent. The problem is we need to get the producer to send some > messages as soon as the system bootstraps (and potentially creates the > topic). So is it possible to know (listener, etc) on the producer side when > the topic is “operational”, or at least to handle a specific exception? > > Or is it a case where the producer will buffer these on the client side? > > Thanks > Cos > > > > On Friday, 8 January 2016 at 16:45, Dana Powers wrote: > > > That is expected when a topic is first created. Once the topic and all > > partitions are initialized, the leader(s) will be available and you can > > produce messages. > > > > Note that if you disable topic auto creation, you should get an > > UnknownTopicOrPartitionError, and you would need to create the topic > > manually. > > > > -Dana > > On Jan 8, 2016 03:20, "Cosmin Marginean" <cosmargin...@gmail.com > (mailto:cosmargin...@gmail.com)> wrote: > > > > > Hi guys > > > > > > I’ve been seeing this in our logs a few times, when initially > > > bootstrapping the system. > > > > > > WARN 2016-01-08 10:59:24,232 [pool-3-thread-11] [none] > > > o.a.kafka.clients.NetworkClient: Error while fetching metadata with > > > correlation id 3 : {mytopic=LEADER_NOT_AVAILABLE} > > > > > > It doesn’t seem to break anything, and it seems to be related to the > fact > > > that we do processing on the topic right after creating it. Below is > our > > > workflow (roughly) > > > * Create topic (using AdminUtils) > > > * Register some tens of consumers > > > * Send some messages with a producer > > > > > > I’d like to know if this is something to worry about (the warning that > is) > > > or if it’s something we should investigate further. > > > > > > Thank you > > > Cosmin > > > > > > > > > > > >
