+1 for adding this either a utility function or cookbook recipe [1].

[1] https://github.com/apache/arrow-cookbook

On Thu, Aug 18, 2022 at 2:34 PM Yaron Gvili <rt...@hotmail.com> wrote:

> I have code in source_node.cc in a local branch adding factories for other
> sources in SourceNode (e.g., streams of RecordBatch, ExecBatch, or
> ArrayVector) which I could make a PR for, if there is interest.
>
>
> Yaron.
> ________________________________
> From: David Li <lidav...@apache.org>
> Sent: Wednesday, August 17, 2022 4:25 PM
> To: dev@arrow.apache.org <dev@arrow.apache.org>
> Subject: Re: [C++] Read Flight data source into Acero
>
> Convert the reader to a RecordBatchReader [1], turn it into an
> AsyncGenerator [2], then wrap it in a SourceNode (though I think SourceNode
> should just expose a helper for wrapping a RecordBatchReader like it
> already does for Tables).
>
> [1]:
> https://github.com/apache/arrow/blob/8474ee5a3ed725d4bb56c75fc1b13a53cba1fd1f/cpp/src/arrow/flight/types.h#L561
> [2]:
> https://github.com/apache/arrow/blob/ee2e9448c8565820ba38a2df9e44ab6055e5df1d/cpp/src/arrow/compute/exec/exec_plan.h#L529-L535
>
> On Wed, Aug 17, 2022, at 15:32, Aldrin wrote:
> > I don't have any pointers, but just wanted to mention that I am going to
> > try and figure this out quite a bit in the next week. I can try to create
> > some relevant cookbook recipes as I plod along.
> >
> > Aldrin Montana
> > Computer Science PhD Student
> > UC Santa Cruz
> >
> >
> > On Wed, Aug 17, 2022 at 9:15 AM Li Jin <ice.xell...@gmail.com> wrote:
> >
> >> Correction: I have a flight::FlightStreamReader (not
> Flight::StreamReader)
> >>
> >> On Wed, Aug 17, 2022 at 12:12 PM Li Jin <ice.xell...@gmail.com> wrote:
> >>
> >> > Hi,
> >> >
> >> > I have a Flight data source (effectively a Flight::StreamReader) and
> I'd
> >> > like to create an Acero source node from it. I wonder if something
> >> already
> >> > exists to do that or if not, perhaps some pointers for me  to take a
> look
> >> > at?
> >> >
> >> > Thanks,
> >> > Li
> >> >
> >> >
> >>
>

Reply via email to