>From my understanding (correct me if I'm wrong here, Jay), if a client does non-blocking writes on a single socket connection, then it's entirely possible to see out-of-order writes in the server from the client's perspective. The requests are serialized/ordered on a connection only until it hits the request channel, after that they may be processed in different threads, and therefore, in an unpredictable order.
I suspect the Node client does indeed do non-blocking sends, but can you confirm this, Ben? - Prashanth On Thu, Dec 13, 2012 at 11:42 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > So my belief is that we handle this correctly and that I was mistaken in > filing that bug. However, since, as the bug points out, the problem can't > exist in the scala client this may be mistaken and a problem may well > remain. > > There are two reasons that requests should have enforced ordering: > 1. The scala client blocks until it gets a response. As you say, this > doesn't help you. > 2. The server only processes a single request per socket at a time. > > Can you confirm that (1) these messages are sent to the same partition > (guarantee is only within a partition), (2) the requests are sent over the > same socket (order can only be guaranteed over a single connection). If > these are true and we are re-ordering I think we have a bug...if so if you > could give a small portable program of some kind that would reproduce the > problem that would be awesome since I don't think the scala client will. > > -Jay > > > On Thu, Dec 13, 2012 at 12:01 AM, ben fleis <ben.fl...@gmail.com> wrote: > > > Hello all, > > > > While running my own system tests against 0.8/HEAD, I was seeing repeated > > ordering failures, and tracked it down a bit further. In simple > summary, I > > am sending out 2 consecutive ProduceRequests, X and Y. Y gets written to > > disk before X. Consumer sees Y before X. > > > > Bug #382 <https://issues.apache.org/jira/browse/KAFKA-382> discusses > this > > possibility, and it appears to describe my issue. My client is, like any > > normal client written in Node, asynchronous by nature. I can add waits > to > > make it synchronous, but this feels like it fails the "best effort" test. > > I.e., it seems reasonable that a pipelining client running without > errors > > ought to maintain order, all the time. > > > > Thoughts? > > > > b > > >
