Re: Writing a client: Connection pooling

2015-05-19 Thread Guozhang Wang
Hi Warren, Now I get your point. Today the request ordering guarantee is enforced on the broker's socket server such that if once a request is read from one socket, the socket key is set to not readable until that request's response is written back to the socket. This behavior could be changed pr

Re: Writing a client: Connection pooling

2015-05-18 Thread Dana Powers
cc: kafka-clients mailing list On May 18, 2015 4:24 PM, "Warren Falk" wrote: > Thanks Guozhang, > > Actually your previous email was clear and I understood it. Current broker > design means that parallel requests require parallel connections. > > But I think you misunderstood me. I am not askin

Re: Writing a client: Connection pooling

2015-05-18 Thread Warren Falk
Thanks Guozhang, Actually your previous email was clear and I understood it. Current broker design means that parallel requests require parallel connections. But I think you misunderstood me. I am not asking if that is how the broker works, now, I am proposing that the broker should not work th

Re: Writing a client: Connection pooling

2015-05-18 Thread Guozhang Wang
Hi Warren, Maybe I was a little misleading in my previous email: 1. The statement that "The server guarantees that on a single TCP connection, requests will be processed in the order" is still valid. On a single connection, all types of requests (produce/fetch/metadata) are handled in order. 2.

Re: Writing a client: Connection pooling

2015-05-14 Thread Warren Falk
The C# client (kafka-net) isn't written by me; I'm just working on it. It has a separate producer and consumer, but the client is designed to connect to each broker exactly once and then reuse (multiplex over) that one connection across all consumers and producers. It's a bit disappointing to see

Re: Writing a client: Connection pooling

2015-05-13 Thread Guozhang Wang
Hello Warren, I seems your C# client is both a producer and a consumer. Then with the behavior of the broker, your suspension is correct that a long pooling fetch using the same TCP connection will block subsequent produce / metadata requests. I think the statement that "it should not generally b

Writing a client: Connection pooling

2015-05-13 Thread Warren Falk
I'm working on the C# client. The current Kafka Protocol page says this: "it should not generally be necessary to maintain multiple connections to a single broker from a single client instance (i.e. connection pooling)" But then says this: "The server guarantees that on a single TCP connection,