Re: Lost of messages at C++ Kafka client

2013-08-07 Thread Jun Rao
Yes, in 0.7, one of the limitations is that the producer doesn't wait for any acknowledgement from the broker. So, if the broker is down, messages still in the client socket buffer may be lost. In 0.8, we added the option so that the producer can wait for an ack, which will help this particular kin

Re: Lost of messages at C++ Kafka client

2013-08-07 Thread Philip O'Toole
If I understand what you are asking, I have dealt successfully with the same type of issue. It can take more than one Boost async_write() over a broken connection before the client software notices that the connection is gone. The best way to detect if a connection is broken is not by detecting th

Lost of messages at C++ Kafka client

2013-08-07 Thread Tianning Zhang
Hi,   I am using Kafka (version 0.7.2) for publishing events from some C++ applications (ca. 20K event /sec). The C++ client is used, which is based on asio. With the asynchronous and batched messaging, one issue I have is that in case Kafka server is broken, the client will not notice the socket

Lost of messages at C++ Kafka client

2013-08-07 Thread Tianning Zhang
Hi, I am using Kafka (version 0.7.2) for publishing events from some C++ applications (ca. 20K event /sec). The C++ client is used, which is based on asio. With the asynchronous and batched messaging, one issue I have is that in case Kafka server is broken, the client will not notice the socket