Yes, in 0.7, one of the limitations is that the producer doesn't wait for any acknowledgement from the broker. So, if the broker is down, messages still in the client socket buffer may be lost. In 0.8, we added the option so that the producer can wait for an ack, which will help this particular kind of data loss.
Thanks, Jun On Wed, Aug 7, 2013 at 2:38 AM, Tianning Zhang <tianning.zh...@zanox.com>wrote: > Hi, > > I am using Kafka (version 0.7.2) for publishing events from some C++ > applications (ca. 20K event /sec). The C++ client is used, which is based > on asio. > With the asynchronous and batched messaging, one issue I have is that in > case Kafka server is broken, the client will not notice the socket lost > immediately. Messages go lost between the moment the socket broke and the > moment when the C++ client issues an error notification. Message lost is > not tolerated in our application. Although we have some proprietary > component to deal with the problem – it is not a convenient solution. > > As I understand, this is a known issue. But I would like to know more > about experiences and best practices for dealing this problems. What can > future versions of Kafka offer to deal with this problems? > > Kind regards > > Tianning Zhang :: tianning.zh...@zanox.com<mailto: > tianning.zh...@zanox.com> > > > > [disclaimer logo: ZANOX.de AG] > > STRAIGHT TO PERFORMANCE > > -------------------------------------------------------------------------------- > > ZANOX.de AG | Headquarters: Berlin AG Charlottenburg | HRB 75459 | VAT > identification number: DE 209981705 > Executive Board: Thomas Joosten (CEO) | Stefanie Lüdecke (CSO) | Joachim > Piroth (CFO) | Christian Rebernik (CTO) > Chairman of the Supervisory Board: Ralph Büchi > > This e-mail and any attachments may contain confidential and/or privileged > information. > If you are not the intended recipient (or have received this e-mail in > error) please notify the sender immediately and delete this e-mail from > your system. > Any other use, copying, disclosure or distribution is strictly forbidden. > > [100 Partnerprogramme 2012] >
