My thinking is ideally the protocol would be more opaque than engineer-y in that an upgrade would happen as part of the negotiation process. For example, when a connection is made, client says "hey, I also support these things" and then server responds and says "hey, let's send data on this channel instead". That way, end consumers don't have to worry about the details of alternative capabilities. (E.g. this is less about a Flight endpoint).
I like the protocol proposal with one small modification: I'd like us to define a default protocol if one is not defined. I think it should probably be grpc+tls but am open to other options. What do others think about a default protocol? On Thu, Mar 28, 2019 at 9:29 PM Wes McKinney <wesmck...@gmail.com> wrote: > hi David, > > This seems like a reasonable evolution from where we are now. I will > defer to others to comment on the low-level details > > This is sort of scope and kind of a can of worms, but one area where > we should invest some thought is alternative FlightData transports, > while allowing the "command layer" to continue to be gRPC. One such > possible alternative scheme includes: > > * gRPC-over-TCP commands (actions, etc.) > * Movement of IPC messages using RDMA (I have never actually used RDMA > but it has been brought up to me as a topic of interest by multiple > parties now) > > If a server supports an alternative protocols (e.g. gRPC-based for > compatibility, plus RDMA for clients that implement it) then perhaps > this information can be encoded in URIs. I'm not sure if there's prior > art to look at on this design-wise > > - Wes > > On Wed, Mar 27, 2019 at 1:24 PM David Li <li.david...@gmail.com> wrote: > > > > Hi all, > > > > We'd like to propose a URI scheme for Flight, in anticipation of > > supporting multiple transports, and different configurations of the > > gRPC transport. This will change Flight.proto[1] in format/ in a > > backwards-incompatible way. This aims to fix ARROW-4651[2]. > > > > The proposal can be found here (it should be commentable by all): > > > https://docs.google.com/document/d/1Eps9eHvBc_qM8nRsTVwVCuWwHoEtQ-a-8Lv5dswuQoM/edit > > > > Any and all feedback is appreciated! > > > > A draft PR is up for Java/C++/Python, though it is far from complete: > > https://github.com/apache/arrow/pull/4047 > > > > [1]: https://github.com/apache/arrow/blob/master/format/Flight.proto > > [2]: https://issues.apache.org/jira/browse/ARROW-4651 > > > > Best, > > David >
