Hi, I am using Kafka 0.7.1, and using the low-level SyncProducer to send messages to a *single* partition from a *single* thread. The client sends messages that contain sequential numbers so it is obvious at the consumer when message order is shuffled. I have noticed that messages can be saved out-or-order by Kafka when there are connection problems, and am looking for possible solutions (I think I already know the cause).
The client sends messages in a retry loop so that it will wait for a short period and then retry to send on any IO errors. In SyncProducer, any IOException triggers a disconnect. Next time send is called a new connection is established. I believe that it is this disconnect/reconnect cycle that can cause messages to be saved to the kafka log in a different order to that of the client. I had previously had the same sort of issue with reconnect.interval/time, which was fixed by disabling those reconnect settings. http://mail-archives.apache.org/mod_mbox/kafka-users/201305.mbox/%3CCAM%2BbZhjssxmUhn_L%3Do0bGsD7PAXFGQHRpOKABcLz29vF3cNOzA%40mail.gmail.com%3E Is there anything in 0.7 that would allow me to solve this problem? The only option I can see at the moment is to not perform retries. Does 0.8 handle this issue any differently? Thanks, Ross
