Re: Arrow based data access

2017-03-23 Thread Jacques Nadeau
Julien, this is a great idea. With the chicken and egg nature of Arrow, having an easy way to connect systems in a generically should help people make progress. I agree with Todd that there is some danger of this being lowest common denominator. That being said, I think it is still a great place to

Re: Arrow based data access

2017-03-19 Thread Todd Lipcon
Hi folks, I'm not sure Kudu integration is a high priority yet for most of us working on the project, but we'd be happy to have contributions from the Arrow community. My worry about the "standard network interface" approach is that it's going to be rather hard to keep it efficient while also giv

Re: Arrow based data access

2017-03-19 Thread Wes McKinney
hi Julien, Having standard RPC/REST messaging protocols for systems to implement sounds like a great idea to me. Some systems might choose to pack Arrow files or streams into a Protocol Buffer or Thrift message, but it would be good to have a "native" protocol for the streaming file format in part