Re: design for Python UDFs in an Ibis/Substrait/Arrow workflow

1. This is probably best discussed on the Ibis mailing list (or Github? I'm not sure how the Ibis community communicates). 2. If we consider "encoding the embedded function in Substrait" to be part of (2) then this can be discussed in the Substrait community. That being said, in the sync call las

Re: Datafusion's Java binding is available in Maven Central

To answer Larry's question, it was simply a matter of timing and also I started the project as a way to learn JNI coding and rust's JNI crate myself so it wasn't until recently that I started to survey better alternatives. I don't really personally have a concrete use case but given enough interest

Re: [VOTE][RUST] Release Apache Arrow Rust 14.0.0 RC1

+1 (non-binding) From: Andrew Lamb Sent: Friday, 13 May 2022 20:05 To: dev Subject: [VOTE][RUST] Release Apache Arrow Rust 14.0.0 RC1 Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 14.0.0. This release candidate is based on c

[RESULT] [VOTE][RUST] Release Apache Arrow Rust 14.0.0 RC1

With 6 +1 votes (3 binding) the release is approved. The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-14.0.0 I have also uploaded them to crates.io: https://crates.io/crates/arrow https://crates.io/crates/arrow-flight https://crates.io/crates/parquet htt

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 8.0.0 RC2

Awesome work Andy -- thank you! On Mon, May 16, 2022 at 9:22 AM Andy Grove wrote: > The crates are now published: > > https://crates.io/crates/datafusion > > https://crates.io/crates/datafusion-cli > > https://crates.io/crates/datafusion-expr > > https://crates.io/crates/datafusion-common > > ht

Feedback on approach for a minimal geospatial library based on Apache Arrow

Hi all, Following the momentum in specifications for storing geospatial vector data using the Arrow columnar format [1][2] and the recent addition of Feather, IPC, and Parquet drivers in GDAL [3], I've been prototyping an R package [4] to add geospatial support to the Arrow R bindings (e.g., regis

Re: [VOTE][RUST] Release Apache Arrow Rust 14.0.0 RC1

+1 (non-binding) Verified on M1 Mac, thanks Andrew! On Sat, May 14, 2022 at 9:09 PM QP Hou wrote: > > +1 (binding) > > On Fri, May 13, 2022 at 11:12 AM Andrew Lamb wrote: > > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, > > version 14.0.0. > > > > This

Re: Datafusion's Java binding is available in Maven Central

On that note, you should be able to use the "jextract" tool from Project Panama to auto generate the glue code and types if you have C headers panama-foreign/panama_jextract.md at foreign-jextract · openjdk/panama-foreign (github.com)

Re: Datafusion's Java binding is available in Maven Central

Hi, Since this is a recent improvement, I'm curious about what motivated the decision to not use the c-data-interface? Was it strictly a matter of timing or familiarity, or is there some advantage to the approach you took? I ask because I'm in the process of moving other JNI interfaces to c-data.

Re: Datafusion's Java binding is available in Maven Central

This is awesome, thank you! On Mon, May 16, 2022 at 6:30 AM Jiayu Liu wrote: > Thanks for the question Atonine, > > So far the data is copied over (not IPC per-se, since it's the same > process), because I haven't found time (and motivation) to migrate to > Arrow C interface just yet. > > A next

Re: design for Python UDFs in an Ibis/Substrait/Arrow workflow

For context, this is a continuation of a previous email discussion: "RPC: Out of Process Python UDFs in Arrow Compute" where we identified reasonable steps to solve are: ostef Process Python UDFs (1) A way to serialize the user function (and function parameters) (in the doc I proposed to use cloudp

RE: [VOTE] [Rust] Move Ballista to new arrow-ballista repository

In general I am +1 on this. However, I believe there is a team who has been actively working on many Ballista enhancements and I don't think that I have seen feedback from them on this yet (I apologize if I missed it). I would be interested in getting their view - in particular on making sure

[VOTE] [Rust] Move Ballista to new arrow-ballista repository

I would like to propose that we move the Ballista project to a new top-level *arrow-ballista* repository. The rationale for this (copied from the GitHub issue [1]) is: - Decouple release process for DataFusion and Ballista - Allow each project to have top-level documentation and use

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 8.0.0 RC2

The crates are now published: https://crates.io/crates/datafusion https://crates.io/crates/datafusion-cli https://crates.io/crates/datafusion-expr https://crates.io/crates/datafusion-common https://crates.io/crates/datafusion-jit https://crates.io/crates/datafusion-physical-expr https://crat

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 8.0.0 RC2

With four +1 votes (three binding) the vote has passed. I will release to crates.io this morning. Thank you to everyone who voted. Andy. On Sun, May 15, 2022 at 7:51 PM Yang hao <1371656737...@gmail.com> wrote: > +1 (non-binding) > Verified on MacOS 12.2 M1 pro > > Best regards, > Remzi > > Fro

Re: Datafusion's Java binding is available in Maven Central

Thanks for the question Atonine, So far the data is copied over (not IPC per-se, since it's the same process), because I haven't found time (and motivation) to migrate to Arrow C interface just yet. A next step, is to allow the project to depend on arrow-c-data [1], and also optimize how .so and