Yes, you could use Acero for this. However, I would hope that someday you
could also use DuckDb and Datafusion to do the combining as well.
In my mind an "engine" is something that takes a plan (Substrait) and zero
or more input streams (Arrow C stream interface[1]) and has one output
stream (Arr
I left some comments on the PR as well...I think this is an important
addition and I'm excited to see this discussion!
If there is further information that needs to be passed along in the
future, schema metadata could be used. Even with schema metadata, the
device type and ID will always need to b
+1 (binding)
Verified on Intel Mac.
Thanks Andy.
On Mon, Apr 10, 2023 at 4:47 PM Andy Grove wrote:
>
> Hi,
>
> I would like to propose a release of Apache Arrow DataFusion Python
> Bindings,
> version 22.0.0.
>
> This release candidate is based on commit:
> 5c90187207e218393297ff3cbe4f08b69049d
Hi,
I would like to propose a release of Apache Arrow DataFusion Python
Bindings,
version 22.0.0.
This release candidate is based on commit:
5c90187207e218393297ff3cbe4f08b69049d224 [1]
The proposed release tarball and signatures are hosted at [2].
The changelog is located at [3].
The Python whee
I am still wrapping my head around some of the technologies so excuse
any ignorance, but seeing as the OP mentioned the use case of /switching
/between execution engines is there not a gap if the concern is more
about /combining/ execution engines? AFAIU Substrait would allow me to
submit diffe
Sorry, I meant:
I am *now* a solid +1
On Mon, Apr 10, 2023 at 1:26 PM Weston Pace wrote:
> I am not a solid +1 and I can see the usefulness. Matt and I spoke on
> this externally and I think Matt has written a great summary. There were a
> few more points that came up in the discussion that I
I am not a solid +1 and I can see the usefulness. Matt and I spoke on this
externally and I think Matt has written a great summary. There were a few
more points that came up in the discussion that I think are particularly
compelling.
* Avoiding device location is generally fatal
In other cases
> There's nothing in the spec today that
prevents users from creating `ArrowDeviceArray` and
`ArrowDeviceArrayStream` themselves
True, but third-party applications aren't going to be the only downstream
users of this API. We also want to build on this within Arrow itself to
enable easier usage of
I suppose I'm a weak +1 in "I don't entirely believe this will be useful
but it doesn't seem harmful". There's nothing in the spec today that
prevents users from creating `ArrowDeviceArray` and
`ArrowDeviceArrayStream` themselves and I'm not familiar enough with the
systems producing / consuming t
The vote passes with five +1 votes (three binding). Thanks to everyone that
helped verify the release.
I'm sorry, I wasn't available to create a new RC with the bug fix this
weekend. Is this a regression since 21.1.0? If so, should we create a patch
release with the fix?
The source release is ava
With 4 +1 votes (3 binding) the release is approved
The release is available here:
https://dist.apache.org/repos/dist/release/arrow/arrow-rs-37.0.0
It has also been released to crates.io
Thank you to everyone who helped verify this release
Raphael
On 10/04/2023 07:46, Wayne Xia wrote:
+1
v
> The ArrowArray struct is not allowed to change, as it would break the ABI:
https://arrow.apache.org/docs/format/CDataInterface.html#updating-this-specification
I was referring more to the future case where we might need to introduce an
`ArrowArrayV2` or something similar precisely because the Ar
+1
verified on x86 linux.
Thanks Andy
On Sat, Apr 8, 2023 at 11:26 PM vin jake wrote:
> +1 (non-binding)
> verified on M1 mac
>
> Thanks Andy.
>
> On Sat, Apr 8, 2023 at 8:42 PM Andrew Lamb wrote:
>
> > +1 (binding)
> > verified on x86 mac
> >
> > If possible it might be nice to cut an RC2 to
13 matches
Mail list logo
