Hi all, I'm working on my own kafka client implementation and I noticed strange situation.
Time(s) | Action 0.0 MetadataRequest 0.0 MetadataResponse 0.2 OffsetRequest 0.2 OffsetResponse 0.3 FetchRequest(MaxWaitTime=20sec) 6.0 ProduceRequest 31.0 FetchResponse ^^^^ notice 25sec gap! 31.0 ProduceResponse What is important, it seems to me that Fetch Request with long poll will block processing of all following requests for the duration of timeout, given that there is no new data. But new data *are* flowing in and Produce Request is waiting right behind the Fetch one in server side processing queue. If what I describe is correct then I see the following problems. 1. From client's point of view, if they have a listener and a sender, then sending data should produce immediate event on listener. 2. If I want clean shutdown of my application within 5sec, but my listeners are configured with 20sec timeout, than I am risking losing the data. My application shutdown time now becomes at least as long as listener's pooling time, which is unreasonable. I do understand why it was implemented that way, probably because of the specification saying ==== "The server guarantees that on a single TCP connection, requests will be processed in the order they are sent and responses will return in that order as well" ==== But to achieve proper functioning of the client, I need to allocate another tcp connection for listeners now? Also, I'm aware about new protocol proposal for the next version of Kafka, would this issue be resolved there? Thanks, Vadim. -- >From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is explicitly specified
