mbutrovich commented on code in PR #1619:
URL: https://github.com/apache/datafusion-comet/pull/1619#discussion_r2031618730
##########
native/core/src/parquet/parquet_exec.rs:
##########
@@ -102,10 +104,13 @@ pub(crate) fn init_datasource_exec(
Ok(Arc::new(DataSourceExec::new(Arc::new(file_scan_config))))
}
-fn get_options(session_timezone: &str) -> (TableParquetOptions,
SparkParquetOptions) {
+fn get_options(
+ session_timezone: &str,
+ pushdown_filters: bool,
+) -> (TableParquetOptions, SparkParquetOptions) {
let mut table_parquet_options = TableParquetOptions::new();
+ table_parquet_options.global.pushdown_filters = pushdown_filters;
// TODO: Maybe these are configs?
Review Comment:
I think we can remove the TODO, and maybe get rid of the `reorder_filters =
true` on the line below. DF still defaults that to false as well so we might
not understand the performance implications of it yet. We could add a config
for that in a follow up PR and measure the difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
