Re: [VOTE] Adopt ADBC database client connectivity specification

2022-10-04 Thread David Li
Just another reminder - this has been open for a while with only one binding vote. I know it's near 10.0.0 and I don't expect this to make the release, but I'd appreciate if we could get this wrapped up one way or another. Thanks, David On Wed, Sep 28, 2022, at 21:55, Steven Deobald wrote: > +1

Re: Integration between ibis-substrait and Acero

2022-10-04 Thread Li Jin
For reference, this is the "relations" entry that I was referring to: https://github.com/apache/arrow/blob/master/python/pyarrow/tests/test_substrait.py#L186 On Tue, Oct 4, 2022 at 3:28 PM Li Jin wrote: > So I made some progress with updated code: > > t = ibis.table([("a", "int64"), ("b"

Re: Integration between ibis-substrait and Acero

2022-10-04 Thread Li Jin
So I made some progress with updated code: t = ibis.table([("a", "int64"), ("b", "int64")], name="table0") test_table_0 = pa.Table.from_pydict({"a": [1, 2, 3], "b": [4, 5, 6]}) result = translate(t, self.compiler) def table_provider(names): if no

Integration between ibis-substrait and Acero

2022-10-04 Thread Li Jin
Hi, I am testing integration between ibis-substrait and Acero but hit a segmentation fault. I think this might be cause the way I am integrating these two libraries are wrong, here is my code: Li Jin 1:51 PM (1 minute ago) to me class BasicTests(unittest.TestCase): """Test basic features"""

Re: [DISCUSS] Apache Iceberg / Apache Hudi support in Arrow

2022-10-04 Thread Will Jones
And to clarify: IIUC FlightSQL is necessary to wrap a Java implementation, since it would be impractical to wrap Java code in the ADBC C interface. But for a Rust / C++ implementation, one could just directly implement ADBC. (Although a FlightSQL implementation would also be useful for distributed

Re: [DISCUSS] Apache Iceberg / Apache Hudi support in Arrow

2022-10-04 Thread David Li
It's possible we could wrap Iceberg et al. in Flight SQL to provide this, exposing Iceberg metadata via the Flight SQL endpoints, and table reads via Substrait plans. (Clients could send Substrait plans through ADBC, and we could integrate ADBC as a type of dataset.) I'm not familiar enough with

Re: [EXTERNAL] Re: [C++][Python]Built-in GRPC health checks in FlightServerBase

2022-10-04 Thread David Li
Ah, to clarify - what I mean is to wrap the gRPC health check, which is itself a normal RPC with a standardized name/signature. So we would add a Ping method to Flight, but have it map to the standard gRPC health check service underneath (and to a custom call when running on UCX). A non-gRPC he