andygrove opened a new issue, #3306: URL: https://github.com/apache/datafusion-comet/issues/3306
## Description `CometScanRule` always uses `native_comet` for V2 scan handling, even when a different scan implementation is configured via `spark.comet.scan.impl`. If the user has configured `native_datafusion` or `native_iceberg_compat`, the V2 path silently ignores that setting and uses `native_comet` instead. ## Expected Behavior If the user has explicitly configured a scan impl other than `native_comet`, and V2 scans only support `native_comet`, the rule should fall back to Spark rather than silently using `native_comet`. ## Context This was discovered while running Spark 3.5.7 SQL tests with `native_datafusion`. The `ParquetV2FilterSuite` and `ParquetV2QuerySuite` tests passed, but only because V2 scans were not actually using the configured `native_datafusion` scan — they were silently falling back to `native_comet`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
