This seems reasonable, however we need to account for existing Flight clients that were written before this.
It seems like the server will need to still handle the ticket returned for getStream() for clients that are unaware of the small result optimization. On Mon, Feb 28, 2022 at 11:26 AM David Li <lidav...@apache.org> wrote: > Ah, that makes more sense, that would be a reasonable extension to Flight > overall. (While we're at it, I think it would help to have an app_metadata > field in FlightInfo as well.) > > On Mon, Feb 28, 2022, at 14:24, Micah Kornfield wrote: > >> > >> But it seems reasonable to add a one-shot query path using DoGet. > > > > > > I was thinking more of adding a bytes field to FlightInfo that could > store > > arrow data. That way GetFlightInfo would be the only RPC necessary for > > small results when executing a CMD. The client doesn't necessarily know > > whether a query will return large or small results. > > > > On Mon, Feb 28, 2022 at 11:04 AM David Li <lidav...@apache.org> wrote: > > > >> I think the focus was on large result sets (though I don't recall this > >> being discussed before) and supporting multi-node setups (hence > >> GetFlightInfo/DoGet are separated). But it seems reasonable to add a > >> one-shot query path using DoGet. > >> > >> On Mon, Feb 28, 2022, at 13:32, Adam Lippai wrote: > >> > I saw the same. A small, stateless query ability would be nice > >> (connection > >> > open, initialization, query in one message, the resultset in the > response > >> > in one message) > >> > > >> > On Mon, Feb 28, 2022, 13:12 Micah Kornfield <emkornfi...@gmail.com> > >> wrote: > >> > > >> >> I'm rereviewing the Flight SQL interfaces, and I'm not sure if I'm > >> missing > >> >> it but is there any optimization for small results? My concern is > that > >> the > >> >> overhead of the RPCs for the DoGet after executing the query could > add > >> >> non-trivial latency for smaller results. > >> >> > >> >> Has anybody else thought about this/investigated it? Am I > understanding > >> >> this correctly? > >> >> > >> >> Thanks, > >> >> Micah > >> >> > >> > -- *James Duong* Lead Software Developer Bit Quill Technologies Inc. Direct: +1.604.562.6082 | jam...@bitquilltech.com https://www.bitquilltech.com This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Thank you.
