timsaucer commented on issue #1103:
URL:
https://github.com/apache/datafusion-python/issues/1103#issuecomment-2931088383
@renato2099 has a very nice PR that addresses the FFI side of this:
https://github.com/apache/datafusion-python/pull/1137 Hopefully we can use that
as a starting point
tespent commented on issue #1103:
URL:
https://github.com/apache/datafusion-python/issues/1103#issuecomment-2830018437
> if you can share, I'd like to learn more about the interplay of the 2
systems.
@aditanase Sure. I think my basic idea is quite similar to yours. But
instead of wr
aditanase commented on issue #1103:
URL:
https://github.com/apache/datafusion-python/issues/1103#issuecomment-2829650344
@tespent I am very intrigued by how you're using Datafusion and ray.data
together - if you can share, I'd like to learn more about the interplay of the
2 systems.
tespent commented on issue #1103:
URL:
https://github.com/apache/datafusion-python/issues/1103#issuecomment-2800371392
> I am concerned about the table providers, though. I think any
implementation will need to get the table provider to provide record batches
efficiently.
A small co
timsaucer commented on issue #1103:
URL:
https://github.com/apache/datafusion-python/issues/1103#issuecomment-2799942428
This is *very* good feedback. I think the catalog provider and schema
provider will be relatively easy to do to provide both pure python and rust-ffi
versions. I am conc
tespent commented on issue #1103:
URL:
https://github.com/apache/datafusion-python/issues/1103#issuecomment-2799841561
@timsaucer This is wonderful! However, I think FFI CatalogProvider is not
enough for my needs, since I'm looking for *pure python-written*
CatalogProvider and SchemaProvid
timsaucer commented on issue #1103:
URL:
https://github.com/apache/datafusion-python/issues/1103#issuecomment-2796689529
This is timely! Now that https://github.com/apache/datafusion/pull/15280 has
landed upstream in DataFusion, I plan on exposing this in `datafusion-python`
in the next re
