kazuyukitanimura commented on code in PR #1390:
URL: https://github.com/apache/datafusion-comet/pull/1390#discussion_r1956799716
##########
common/src/main/scala/org/apache/comet/CometConf.scala:
##########
@@ -216,6 +216,17 @@ object CometConf extends ShimCometConf {
val COMET_EXEC_INITCAP_ENABLED: ConfigEntry[Boolean] =
createExecEnabledConfig("initCap", defaultValue = false)
+ val COMET_EXEC_AGGREGATE_ENFORCE_RESULTS: ConfigEntry[Boolean] =
+ conf("spark.comet.exec.aggregate.enforceResults")
+ .doc("Whether to enforce converting results in the Final stage of a
HashAggregate, " +
+ "When enabled, Final-mode hashAggregates will not be converted to
Comet, this can cause " +
+ "issues when native shuffle is enabled. " +
+ "If this is disabled, unsupported result expressions will be " +
+ "separated into a ProjectExec to allow HashAggregate to complete
natively. " +
+ "This is disabled by default.")
Review Comment:
I would like to understand this more
`Final-mode hashAggregates will not be converted to Comet` so final
aggregation falls back to Spark?
`this can cause isues when native shuffle is enabled.` why it is the case?
And when should we enable this option?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
