adriangb commented on issue #15037: URL: https://github.com/apache/datafusion/issues/15037#issuecomment-2763348634
wrt waiting for filter pushdown to be enabled by default, I think we're just making our lives harder by coupling them, especially since we can already test them together under a feature flag. I also would like to leverage this work to justify a lot of other optimizations: - piping through file level stats and using those to prune with dynamic filters -> prune without even reading parquet metadata - roughly ordering files by the sort order based on stats -> this optimization becomes even more efficient - resolve merge conflicts and take next steps with per-file filters This is unfortunately a blocker for all of that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
