On Fri, Jun 30, 2017 at 2:46 PM, Maurice Barnum <m...@yahoo-inc.com.invalid> wrote:
> it's a bit unfortunate the proxy would have to decode enough of the wire > protocol to find the lookup commands and responses to build the "routing > map". i'd still be tempted to do it with nginx and a lua module. if you're > careful, you can avoid deserializing responses unless you have an > outstanding lookup request, but you'll still have to keep track of the > message framing. > But the deserialization is only required for topic and partitions metadata lookups. In general, and that will be true with the proxy as well, the TCP connection used for lookup is different from the TCP connections used to send/receive data. That's because the lookups all happen on the "serviceUrl" hostname, while the send/receive is done to the direct broker. With the proxy, on the connection where the actual data is being sent, we just need to deserialize the initial Connect/Connected commands, and after that remove the deserialization logic. That makes the proxy very cheap in terms of memory allocated (GC rate is only 1 object per buffer transferred each way) and CPU. -- Matteo Merli <matteo.me...@gmail.com>
