ost cases, assuming the
> (Substrait-consumer-side) reproduced column-names are natural/standardized.
> For example, in the cast_table example, the user would not need to specify
> "value1" and "value2" as output field-names for cast_table because these
> natural names
ample, in the cast_table example, the user would not need to specify
> "value1" and "value2" as output field-names for cast_table because these
> natural names would be the default.
>
>
> Yaron.
> ____
> From: Jeroen van Stra
From: Jeroen van Straten
Sent: Wednesday, April 20, 2022 11:54 AM
To: dev@arrow.apache.org
Subject: Re: [C++] output field names in Arrow Substrait
> > The names at the relation root are a different story, since they specify
> > the column names for the produced table, but t
t;>> key
> >>> > > that
> >>> > > > exists in all tables. This is a use case where the key should
> (not
> >>> > must)
> >>> > > be
> >>> > > > specified by name for convenience.
> >>> > > > In b
ying
>>> > > > a string-name as a parameter to a relation that would later get
>>> passed
>>> > to
>>> > > > an Arrow execution node in an options instance and used to
>>> dynamically
>>> > > set
>>> > > > up field-names. This strin
> > > > in the Substrait plan are output field-names of intermediate
>> relations;
>> > > > this in itself is not a problem, because these field-names can (in
>> > > > principle) be recomputed to allow them to be matched to the
>> dynamically
>> > > >
ts Substrait module) ends up with non-natural field-names like
> > > > "FieldPath(1)" that fail to be matched.
> > > >
> > > > That's why this is not a Substrait specific problem, but one that is
> > > > related to Ibis, Ibis-Substrait, and Arr
roblem, but one that is
> > > related to Ibis, Ibis-Substrait, and Arrow together. We think it can be
> > > resolved either by changing Substrait to assist Arrow by specifying the
> > > field-names of intermediate relations in the plan, so that they are
>
schemata before options and node instances are created. We
> > might come up with other solutions in this discussion, of course. Either
> > way, for this second use case, the field-names of intermediate relations
> > must be natural; they cannot be left as something like "Fi
Either
> way, for this second use case, the field-names of intermediate relations
> must be natural; they cannot be left as something like "FieldPath(1)".
>
>
> Yaron.
> ____
> From: Weston Pace
> Sent: Tuesday, April 19, 2022 7:12 PM
field-names of intermediate relations must be natural; they cannot be left as
something like "FieldPath(1)".
Yaron.
From: Weston Pace
Sent: Tuesday, April 19, 2022 7:12 PM
To: dev@arrow.apache.org
Cc: Li Jin
Subject: Re: [C++] output field names in Arrow
user writing an Ibis expression and the remaining steps -
> Substrait compilation, serialization to protobuf,, deserialization to an
> Arrow plan, and execution of the plan - are done systematically.
>
>
> Yaron.
>
> From: Weston Pace
> Sent: Tuesday, April 19, 2022
___
From: Weston Pace
Sent: Tuesday, April 19, 2022 1:01 PM
To: dev@arrow.apache.org
Cc: Li Jin
Subject: Re: [C++] output field names in Arrow Substrait
Hi Yaron, I think you might have forgotten the links for [1][2][3] so
I'm not entirely sure of the context. Are you going from Sub
Hi Yaron, I think you might have forgotten the links for [1][2][3] so
I'm not entirely sure of the context. Are you going from Substrait to
an Arrow execution plan? Or are you going from an Arrow execution
plan to Substrait?
For Substrait -> Arrow most of our execution nodes should take in a
Fie
14 matches
Mail list logo
