Agreed with Antoine on grpc+tcp as the default. A gRPC server generally won't offer both encrypted and unencrypted connections, so this won't establish an insecure session where a secure one is available. We could implement a TLS upgrade mechanism later as well. I've updated the document to match.
I'll also note that at least in preliminary Java benchmarks we conducted, TLS had a very significant throughput cost in gRPC/Flight (enough to wipe out any improvement Flight gave). Best, David On 4/2/19, Antoine Pitrou <anto...@python.org> wrote: > > Le 02/04/2019 à 01:28, Jacques Nadeau a écrit : >> My thinking is ideally the protocol would be more opaque than engineer-y >> in >> that an upgrade would happen as part of the negotiation process. For >> example, when a connection is made, client says "hey, I also support >> these >> things" and then server responds and says "hey, let's send data on this >> channel instead". That way, end consumers don't have to worry about the >> details of alternative capabilities. (E.g. this is less about a Flight >> endpoint). >> >> I like the protocol proposal with one small modification: I'd like us to >> define a default protocol if one is not defined. I think it should >> probably >> be grpc+tls but am open to other options. What do others think about a >> default protocol? > > That sounds reasonable, though IMHO it may be more useful if the default > is the unencrypted version. > > (side note: while symmetric encryption is fast nowadays, it's not > entirely costless) > > Regards > > Antoine. > > >> >> >> >> On Thu, Mar 28, 2019 at 9:29 PM Wes McKinney <wesmck...@gmail.com> wrote: >> >>> hi David, >>> >>> This seems like a reasonable evolution from where we are now. I will >>> defer to others to comment on the low-level details >>> >>> This is sort of scope and kind of a can of worms, but one area where >>> we should invest some thought is alternative FlightData transports, >>> while allowing the "command layer" to continue to be gRPC. One such >>> possible alternative scheme includes: >>> >>> * gRPC-over-TCP commands (actions, etc.) >>> * Movement of IPC messages using RDMA (I have never actually used RDMA >>> but it has been brought up to me as a topic of interest by multiple >>> parties now) >>> >>> If a server supports an alternative protocols (e.g. gRPC-based for >>> compatibility, plus RDMA for clients that implement it) then perhaps >>> this information can be encoded in URIs. I'm not sure if there's prior >>> art to look at on this design-wise >>> >>> - Wes >>> >>> On Wed, Mar 27, 2019 at 1:24 PM David Li <li.david...@gmail.com> wrote: >>>> >>>> Hi all, >>>> >>>> We'd like to propose a URI scheme for Flight, in anticipation of >>>> supporting multiple transports, and different configurations of the >>>> gRPC transport. This will change Flight.proto[1] in format/ in a >>>> backwards-incompatible way. This aims to fix ARROW-4651[2]. >>>> >>>> The proposal can be found here (it should be commentable by all): >>>> >>> https://docs.google.com/document/d/1Eps9eHvBc_qM8nRsTVwVCuWwHoEtQ-a-8Lv5dswuQoM/edit >>>> >>>> Any and all feedback is appreciated! >>>> >>>> A draft PR is up for Java/C++/Python, though it is far from complete: >>>> https://github.com/apache/arrow/pull/4047 >>>> >>>> [1]: https://github.com/apache/arrow/blob/master/format/Flight.proto >>>> [2]: https://issues.apache.org/jira/browse/ARROW-4651 >>>> >>>> Best, >>>> David >>> >> >
