Jason Rosenberg created KAFKA-1124:
--------------------------------------
Summary: Sending to a new topic (with auto.create.topics.enable)
returns ERROR
Key: KAFKA-1124
URL: https://issues.apache.org/jira/browse/KAFKA-1124
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8
Reporter: Jason Rosenberg
I had thought this was reported issue, but can't seem to find a previous report
for it.
If auto.create.topics.enable is true, a producer still gets an ERROR logged on
the first attempt to send a message to a new topic, e.g.:
2013-11-06 03:00:08,638 ERROR [Thread-1] async.DefaultEventHandler - Failed to
collate messages by topic, partition due to: Failed to fetch topic metadata for
topic: mynewtopic
2013-11-06 03:00:08,638 INFO [Thread-1] async.DefaultEventHandler - Back off
for 100 ms before retrying send. Remaining retries = 3
This usually clears itself up immediately on retry (after 100 ms), as handled
by the the kafka.producer.async.DefaultEventHandler (with retries enabled).
However, this is logged to the client as an ERROR, and looks scary, when in
fact it should have been a normal operation (since we have
auto.create.topics.enable=true).
There should be a better interaction here between the producer client and the
server.
Perhaps the server can create the topic in flight before returning the metadata
request.
Or, if it needs to be asynchronous, it could return a code which indicates
something like: "The topic doesn't exist yet, it is being created, try again
shortly".....and have the client automatically retry (even if retries not
enabled, since it's not an ERROR condition, really).
The ERROR log level is a problem since apps often have alert systems set up to
notify when any ERROR happens, etc.....
--
This message was sent by Atlassian JIRA
(v6.1#6144)
