Handling of failure to send message in kafka async producer

[Prashant Prakash]

Dear Kafka Users,

We are using kafka 0.8.0 is our application development. To keep message
delivery reliable we want to detect any failure while sending message. To
get high throughput we are using async producer.

As of kafka 0.8.0 async producer implementation, failure to send to message
is logged but not thrown back to producer client.

Snippet from ProducerSendThread -

def tryToHandle(events: Seq[KeyedMessage[K,V]]) {
    val size = events.size
    try {
      debug("Handling " + size + " events")
      if(size > 0)
        handler.handle(events)
    }catch {
      case e: Throwable => error("Error in handling batch of " + size + "
events", e)
    }
  }

Is there any better solution to monitor the async send failure ?

Few of the solution we came up are :
1. Create message buffer in our application logic. Instantiate configurable
pool of  sync producer which reads and send message from buffer. (Sync
producer throws back exception to the caller class)
2. Extend DefaultEventHandler and pass an exception handler callback. On
encountering exception the callback  method -handleException() will be
called notifying application code of any failure.

Please share views, solutions which you have used to tack async send
failure.

Thank you very much.

Best regards
Prashant