So long as you set max.inflight.requests.per.connection = 1 Kafka should provide strong ordering within a partition (so use the same key for messages that should retain their order). There is a bug currently raised agaisnt this feature though where there is an edge case that can cause ordering issues.
https://issues.apache.org/jira/browse/KAFKA-3197 > On 17 Feb 2016, at 07:17, Ivan Dyachkov <d...@dyachkov.org> wrote: > > Hello all. > > I'm developing a kafka client and have a question about kafka server > guarantees. > > A statement from > https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Network > makes me a bit confused: > > "The server guarantees that on a single TCP connection, requests will be > processed in the order they are sent and responses will return in that order > as well. The broker's request processing allows only a single in-flight > request per connection in order to guarantee this ordering. Note that clients > can (and ideally should) use non-blocking IO to implement request pipelining > and achieve higher throughput. i.e., clients can send requests even while > awaiting responses for preceding requests since the outstanding requests will > be buffered in the underlying OS socket buffer. All requests are initiated by > the client, and result in a corresponding response message from the server > except where noted." > > Does this mean that when a client is sending more than one in-flight request > per connection, the server does not guarantee that responses will be sent in > the same order as requests? > > In other words, if I have a strictly monotonically increasing integer as a > correlation id for all requests, can I rely on Kafka that correlation id in > responses will also have this property? > > Thanks. > > /Ivan
