So to expand on that, as a result my belief is that since we only allow one outstanding request per socket at a time it should be impossible to re-order requests. But we haven't rigorously tested this. If someone has a script that doesn't use the scala client and makes lots of requests I think we could test this easily. If not I can write a test that directly uses a socket and the network server to validate this, which might be a good thing to do anyway.
-Jay On Thu, Dec 13, 2012 at 11:06 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > That is correct, but the network server only enqueues a single request > from each socket at any given time. > > -Jay > > > On Thu, Dec 13, 2012 at 9:40 AM, Prashanth Menon < > prashanth.men...@gmail.com> wrote: > >> From my understanding (correct me if I'm wrong here, Jay), if a client >> does >> non-blocking writes on a single socket connection, then it's entirely >> possible to see out-of-order writes in the server from the client's >> perspective. The requests are serialized/ordered on a connection only >> until it hits the request channel, after that they may be processed in >> different threads, and therefore, in an unpredictable order. >> >> I suspect the Node client does indeed do non-blocking sends, but can you >> confirm this, Ben? >> >> - Prashanth >> >> On Thu, Dec 13, 2012 at 11:42 AM, Jay Kreps <jay.kr...@gmail.com> wrote: >> >> > So my belief is that we handle this correctly and that I was mistaken in >> > filing that bug. However, since, as the bug points out, the problem >> can't >> > exist in the scala client this may be mistaken and a problem may well >> > remain. >> > >> > There are two reasons that requests should have enforced ordering: >> > 1. The scala client blocks until it gets a response. As you say, this >> > doesn't help you. >> > 2. The server only processes a single request per socket at a time. >> > >> > Can you confirm that (1) these messages are sent to the same partition >> > (guarantee is only within a partition), (2) the requests are sent over >> the >> > same socket (order can only be guaranteed over a single connection). If >> > these are true and we are re-ordering I think we have a bug...if so if >> you >> > could give a small portable program of some kind that would reproduce >> the >> > problem that would be awesome since I don't think the scala client will. >> > >> > -Jay >> > >> > >> > On Thu, Dec 13, 2012 at 12:01 AM, ben fleis <ben.fl...@gmail.com> >> wrote: >> > >> > > Hello all, >> > > >> > > While running my own system tests against 0.8/HEAD, I was seeing >> repeated >> > > ordering failures, and tracked it down a bit further. In simple >> > summary, I >> > > am sending out 2 consecutive ProduceRequests, X and Y. Y gets >> written to >> > > disk before X. Consumer sees Y before X. >> > > >> > > Bug #382 <https://issues.apache.org/jira/browse/KAFKA-382> discusses >> > this >> > > possibility, and it appears to describe my issue. My client is, like >> any >> > > normal client written in Node, asynchronous by nature. I can add >> waits >> > to >> > > make it synchronous, but this feels like it fails the "best effort" >> test. >> > > I.e., it seems reasonable that a pipelining client running without >> > errors >> > > ought to maintain order, all the time. >> > > >> > > Thoughts? >> > > >> > > b >> > > >> > >> > >
