I started a vote for the other Flight discussion thread, which will close on Friday. Since I'm about to leave on vacation can Antoine or Jacques run the vote for this one?
Thanks On Tue, Apr 2, 2019 at 7:07 AM David Li <li.david...@gmail.com> wrote: > > Agreed with Antoine on grpc+tcp as the default. A gRPC server > generally won't offer both encrypted and unencrypted connections, so > this won't establish an insecure session where a secure one is > available. We could implement a TLS upgrade mechanism later as well. > I've updated the document to match. > > I'll also note that at least in preliminary Java benchmarks we > conducted, TLS had a very significant throughput cost in gRPC/Flight > (enough to wipe out any improvement Flight gave). > > Best, > David > > On 4/2/19, Antoine Pitrou <anto...@python.org> wrote: > > > > Le 02/04/2019 à 01:28, Jacques Nadeau a écrit : > >> My thinking is ideally the protocol would be more opaque than engineer-y > >> in > >> that an upgrade would happen as part of the negotiation process. For > >> example, when a connection is made, client says "hey, I also support > >> these > >> things" and then server responds and says "hey, let's send data on this > >> channel instead". That way, end consumers don't have to worry about the > >> details of alternative capabilities. (E.g. this is less about a Flight > >> endpoint). > >> > >> I like the protocol proposal with one small modification: I'd like us to > >> define a default protocol if one is not defined. I think it should > >> probably > >> be grpc+tls but am open to other options. What do others think about a > >> default protocol? > > > > That sounds reasonable, though IMHO it may be more useful if the default > > is the unencrypted version. > > > > (side note: while symmetric encryption is fast nowadays, it's not > > entirely costless) > > > > Regards > > > > Antoine. > > > > > >> > >> > >> > >> On Thu, Mar 28, 2019 at 9:29 PM Wes McKinney <wesmck...@gmail.com> wrote: > >> > >>> hi David, > >>> > >>> This seems like a reasonable evolution from where we are now. I will > >>> defer to others to comment on the low-level details > >>> > >>> This is sort of scope and kind of a can of worms, but one area where > >>> we should invest some thought is alternative FlightData transports, > >>> while allowing the "command layer" to continue to be gRPC. One such > >>> possible alternative scheme includes: > >>> > >>> * gRPC-over-TCP commands (actions, etc.) > >>> * Movement of IPC messages using RDMA (I have never actually used RDMA > >>> but it has been brought up to me as a topic of interest by multiple > >>> parties now) > >>> > >>> If a server supports an alternative protocols (e.g. gRPC-based for > >>> compatibility, plus RDMA for clients that implement it) then perhaps > >>> this information can be encoded in URIs. I'm not sure if there's prior > >>> art to look at on this design-wise > >>> > >>> - Wes > >>> > >>> On Wed, Mar 27, 2019 at 1:24 PM David Li <li.david...@gmail.com> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> We'd like to propose a URI scheme for Flight, in anticipation of > >>>> supporting multiple transports, and different configurations of the > >>>> gRPC transport. This will change Flight.proto[1] in format/ in a > >>>> backwards-incompatible way. This aims to fix ARROW-4651[2]. > >>>> > >>>> The proposal can be found here (it should be commentable by all): > >>>> > >>> https://docs.google.com/document/d/1Eps9eHvBc_qM8nRsTVwVCuWwHoEtQ-a-8Lv5dswuQoM/edit > >>>> > >>>> Any and all feedback is appreciated! > >>>> > >>>> A draft PR is up for Java/C++/Python, though it is far from complete: > >>>> https://github.com/apache/arrow/pull/4047 > >>>> > >>>> [1]: https://github.com/apache/arrow/blob/master/format/Flight.proto > >>>> [2]: https://issues.apache.org/jira/browse/ARROW-4651 > >>>> > >>>> Best, > >>>> David > >>> > >> > >
