If you are using new Java producer, reorder could happen if
max.inflight.requests.per.connection is set to > 1 and retries are enabled
- which are both default settings.

Can you set max.in.flight.requests.per.connection to 1 and see if this
solve the issue?

Jiangjie (Becket) Qin

On 5/12/15, 12:57 PM, "Scott Chapman" <sc...@woofplanet.com> wrote:

>We are using the Java producer API (0.8.2.1 if I am not mistaken). We are
>using producer type of sync though.
>
>On Tue, May 12, 2015 at 3:50 PM Magnus Edenhill <mag...@edenhill.se>
>wrote:
>
>> Hi Scott,
>>
>> what producer client are you using?
>>
>> Reordering is possible in async producers in the case of temporary
>>broker
>> failures
>> and the combination of request.required.acks != 0 and retries > 0.
>>
>> Consider the case where a producer has 20 messages in-flight to the
>>broker,
>> out of those
>> messages # 1-10 fails due to some temporary failure (?) on the broker
>>side,
>> but message # 11-20 are accepted.
>> When the producer receives error results from the broker for message #
>>1-10
>> it will try to resend
>> these 10 failed messages, that are now accepted, causing them to end up
>> after message #20 in the log - thus reordered.
>>
>> This failure scenario should be rather rare though.
>>
>>
>> Regards,
>> Magnus
>>
>> 2015-05-12 20:18 GMT+02:00 Scott Chapman <sc...@woofplanet.com>:
>>
>> > We are basically using kafka as a transport mechanism for multi-line
>>log
>> > files.
>> >
>> > So, for this we are using single partition topics (with a replica for
>> good
>> > measure) writing to a multi-broker cluster.
>> >
>> > Our producer basically reads a file line-by-line (as it is being
>>written
>> > to) and publishes each line as a message to the topic. We are also
>> writing
>> > as quickly as we can (not waiting for ACK).
>> >
>> > What I am seeing is occasionally the messages in the topic appear to
>>be
>> > slightly out of order when compared to the source file they were based
>> on.
>> >
>> > I am wonder if this might happen when the producer switches brokers
>> because
>> > we are not waiting for the ACK before continuing to write.
>> >
>> > Does this make any sense??
>> >
>> > Thanks in advance!
>> >
>> > -Scott
>> >
>>

Reply via email to