Hi,

I am using Kafka 0.7.1, and using the low-level SyncProducer to send
messages to a *single* partition from a *single* thread.
The client sends messages that contain sequential numbers so it is obvious
at the consumer when message order is shuffled.
I have noticed that messages can be saved out-or-order by Kafka when there
are connection problems, and am looking for possible solutions (I think I
already know the cause).

The client sends messages in a retry loop so that it will wait for a short
period and then retry to send on any IO errors.  In SyncProducer, any
IOException triggers a disconnect.  Next time send is called a new
connection is established.  I believe that it is this disconnect/reconnect
cycle that can cause messages to be saved to the kafka log in a different
order to that of the client.

I had previously had the same sort of issue with reconnect.interval/time,
which was fixed by disabling those reconnect settings.
http://mail-archives.apache.org/mod_mbox/kafka-users/201305.mbox/%3CCAM%2BbZhjssxmUhn_L%3Do0bGsD7PAXFGQHRpOKABcLz29vF3cNOzA%40mail.gmail.com%3E

Is there anything in 0.7 that would allow me to solve this problem?  The
only option I can see at the moment is to not perform retries.

Does 0.8 handle this issue any differently?

Thanks,
Ross

Reply via email to