On Thu, Feb 25, 2016 at 12:05 PM, Henry Robinson <he...@cloudera.com> wrote: > The way I'm thinking about is that someone upstream makes a Kudu-specific > request, but as part of that request provides a descriptor of a shared > ring-buffer. Reading Arrow batches from and writing to that buffer is part > of a simple standard protocol. I agree there's little justification for a > general "request data from data source" protocol. > > Or does Kudu have very specific stuff in the "get next batch" API? >
The "continue scan" RPC is fairly simple. The kudu-specific stuff we have is a sequence ID (so that if a request gets lost or reordered we don't end up giving back incorrect results), a batch size, and a flag to close the current scanner. We also have a separate 'keepalive scan' RPC which the client sends if it's not yet ready for more data, but it wants to keep the stream alive. So, I suppose it would be reasonable to use the "start scan" request to establish an Arrow "stream" of sorts. We do need some kind of liveness indicator from the client to the server so that the server knows when it's safe to release resources, etc. > >> >> For the use case of external UDFs or UDTFs, I think there's good >> opportunity to standardize, though - that interface is a much more >> "pure functional interface" where we can reasonably expect everyone to >> conform to the same protocol. >> >> Regarding RPC overhead, so long as the batches are reasonable size, I >> think it could be well amortized. Kudu's RPC system can do 300k >> RPCs/sec or more on localhost, so if each one of those RPCs >> corresponds to an 8MB data buffer, that's 2.4TB/sec of bandwidth >> (about 100x more than RAM bandwidth). Or put another way, with 8MB >> batches, assuming 24GB/sec RAM bandwidth, you only need 3k round >> trips/second to saturate the bus. Even a super slow RPC control plane >> should be able to handle that in its sleep. >> > > Fair enough. What about the latency of those RPCs? > On my laptop with a single-threaded client making back to back RPCs to localhost I get: I0225 12:12:08.943912 14968 rpc-bench.cc:124] Reqs/sec: 28134.4 I0225 12:12:08.943958 14968 rpc-bench.cc:125] User CPU per req: 16.916us I0225 12:12:08.943967 14968 rpc-bench.cc:126] Sys CPU per req: 28.8567us so that implies about 35us round trip latency. It could probably be improved a bit with some effort, but this has never been the majority of our request processing time. Other systems might be a bit different, but I'd expect <100us of even a "slow" RPC system. -- Todd Lipcon Software Engineer, Cloudera
