vmingchen commented on code in PR #66:
URL: https://github.com/apache/datafusion-ray/pull/66#discussion_r1973563759
##########
src/util.rs:
##########
@@ -397,6 +402,52 @@ fn print_node(plan: &Arc<dyn ExecutionPlan>, indent:
usize, output: &mut String)
}
}
+async fn exec_sql(query: String, tables: Vec<(String, String)>) ->
PyResult<RecordBatch> {
+ let ctx = SessionContext::new();
+ for (name, path) in tables {
+ if path.ends_with(".parquet") {
Review Comment:
Thanks for pointing this out! I looked into it, and it turned out
[`register_parquet` internally use `register_listing_table` as
well](https://github.com/apache/datafusion/blob/fc2fbb3d6b3aded73f1b0902168e008e580c89c1/datafusion/core/src/execution/context/parquet.rs#L55-L61).
The later is capable of register both a single file and a directory of files.
So I have changed the function to use `register_listing_table` only in
https://github.com/apache/datafusion-ray/pull/66/commits/b8e0c6b3c10321896f39e33dee94ea843d91505a;
the new commit also adds a unit test to check that it is working for both file
and directory and a doc pointing to the format of the URI.
Please take another look; thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
