timsaucer commented on issue #1103: URL: https://github.com/apache/datafusion-python/issues/1103#issuecomment-2799942428
This is *very* good feedback. I think the catalog provider and schema provider will be relatively easy to do to provide both pure python and rust-ffi versions. I am concerned about the table providers, though. I think any implementation will need to get the table provider to provide record batches efficiently. I guess that means your python implementation will return pyarrow record batch reader. Is there any way your table provider could have a rust wrapper that we could expose to python? If the source is open, can you show an example of where it is coming from? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
