wForget commented on code in PR #3060:
URL: https://github.com/apache/datafusion-comet/pull/3060#discussion_r2680638947
##########
spark/src/main/scala/org/apache/comet/serde/operator/CometNativeScan.scala:
##########
@@ -191,6 +193,10 @@ object CometNativeScan extends
CometOperatorSerde[CometScanExec] with Logging {
}
}
+ // Add runtime filter bounds if available
+ // These are pushed down from join operators to enable I/O reduction
+ addRuntimeFilterBounds(scan, nativeScanBuilder)
Review Comment:
> Filters are pushed down but native filtering is not implemented, right ? I
think those fallback to spark.
There are several possibilities:
1. Runtime Filters have been pushed down to the native parquet reader, but
the parquet reader is not enabled or does not support them. (Parquet row group
filters will be ineffective, but subsequent Filter operator will be effective.)
2. Runtime Filters have been generated, but they caused the spark plan to
fall back to vanilla spark. (As far as I know, Comet’s support for subqueries
may not be complete.)
3. Runtime Filters were not generated correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
