I am interested in this very topic as well. Also, can the trunk version of the producer be used with an existing 0.8.1.1 broker installation, or does one need to wait for 0.8.2 (at least)?
Thanks, -Jonathan On Aug 26, 2014, at 12:35 PM, Ryan Persaud <ryan_pers...@symantec.com> wrote: > Hello, > > I'm looking to insert log lines from log files into kafka, but I'm concerned > with handling asynchronous send() failures. Specifically, if some of the log > lines fail to send, I want to be notified of the failure so that I can > attempt to resend them. > > Based on previous threads on the mailing list > (http://comments.gmane.org/gmane.comp.apache.kafka.user/1322), I know that > the trunk version of kafka supports callbacks for dealing with failures. > However, the callback function is not passed any metadata that can be used by > the producer end to reference the original message. Including the key of the > message in the RecordMetadata seems like it would be really useful for > recovery purposes. Is anyone using the callback functionality to trigger > resends of failed messages? If so, how are they tying the callbacks to > messages? Is anyone using other methods for handling async errors/resending > today? I can’t imagine that I am the only one trying to do this. I asked > this question on the IRC channel today, and it sparked some discussion, but I > wanted to hear from a wider audience. > > Thanks for the information, > -Ryan >
