but then the comment is not right? since not all flights will be consumed?
"To consume the whole
> > > * flight, all endpoints must be consumed."
if we introduce a itinerary concept, we have a complete story and mental
model. something like
message FlightGetInfo {
// schema of the dataset as described in Schema.fbs::Schema.
bytes schema = 1;
/*
* The descriptor associated with this info.
*/
FlightDescriptor flight_descriptor = 2;
/*
* A list of endpoints associated with the flight. To consume the whole
* flight, all endpoints must be consumed.
*/
repeated FlightItinerary itinerary = 3;
// Set these to -1 if unknown.
int64 total_records = 4;
int64 total_bytes = 5;
}
message FlightItinerary {
/*
* A list of endpoints associated with the itinerary. To consume the whole
* itinerary, all endpoints must be consumed.
*/
repeated FlightEndpoint endpoint = 1;
}
On Wed, Apr 3, 2019 at 12:40 AM Wes McKinney <wesmck...@gmail.com> wrote:
> On Tue, Apr 2, 2019 at 10:08 PM ming zhang <ming.zhang.ch...@gmail.com>
> wrote:
> >
> > in a case where there are multiple ways to retrieve this logical data
> set,
> > how to represent this in the response?
> >
> > for example, assume there is a data set that has
> > part 1 in endpoint 1 and part 2 in endpoint 2 with tcp as transport
> > both part 1 and part 2 in endpoint 3, with infiniband as transport
> >
> > now how we return this back to client so client decide which one to
> consume?
> >
>
> One way to do this would be to have the Ticket payload contain a
> serialized object (e.g. a protocol buffer) which contains this
> additional metadata for the client to decide which endpoints to
> access.
>
> The Ticket returned in FlightEndpoint is opaque data so can contain
> anything. In our performance test server the Ticket is a protobuf:
>
>
> https://github.com/apache/arrow/blob/master/cpp/src/arrow/flight/perf-server.cc#L169
>
> > the mental model i have with FightService is similar to travel with
> flight.
> >
> > - a travel agency lists many travel choices.
> > - for a trip from NYC to SFO, there are different itinerary, like
> NYC->SFO
> > directly. NYC->Chicago -> SFO, and so on.
> > - for each itinerary, there could be one or more logical parts (hop)
> > - for each hop, there are different ways, like business class, economic
> > class, etc
> > - they all need a ticket to finish the trip.
> >
> > is this right, or way off?
> >
> > Thanks
> > Ming
> >
> >
> > On Tue, Apr 2, 2019 at 9:59 PM Wes McKinney <wesmck...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > A FlightGetInfo plan corresponds to a single logical dataset. The
> > > dataset may be spread across multiple endpoints, so if you want the
> > > whole dataset you have to execute DoGet against them all.
> > >
> > > I'm not sure what you mean by "provide more than one execution plan".
> > >
> > > - Wes
> > >
> > > On Tue, Apr 2, 2019 at 7:54 PM ming zhang <ming.zhang.ch...@gmail.com>
> > > wrote:
> > > >
> > > > Hi All
> > > >
> > > > Wonder how to understand this comment?
> > > >
> > > > Looks like this assume we only have one "itinerary" and finish it
> need to
> > > > consume all the flight endpoints.
> > > >
> > > > /*
> > > > * A list of endpoints associated with the flight. To consume the
> whole
> > > > * flight, all endpoints must be consumed.
> > > > */
> > > >
> > > > What if we do want to provide more than one execution plan
> (itinerary)
> > > and
> > > > each have multiple steps?
> > > >
> > > > Thanks
> > > > Ming
> > >
>
