sunchao commented on PR #5533: URL: https://github.com/apache/datafusion-comet/pull/5533#issuecomment-5803997435
@andygrove Addressed in c7c70faf46d7fed8131088e2f999166c7306ffdf, rebased onto `main` at dd68a531c. Removed `restoreNativeAggregateBuffers` and the extra restoration walk from `preserveEvaluationMasks`. The rule keeps early Final tagging so fresh partials choose compatible buffers before AQE materialization, then relies on the existing `revertUnsafePartialAggregates` / `restoreSparkPartial` path from #5421 for restoration. Those upstream helpers are unchanged, and the remaining predicate uses `allAggsSupportNativePartialToSparkFinal`. Both AQE aggregate regressions passed, including the fresh/reused native-plan matrix and the actual join change. The focused evaluation-mask, rule, Celeborn, and SQL tests passed: 106 executed tests on Spark 4.1.3 (107 successful registrations including one Spark-3.5-gated body). `cargo build --locked`, JVM compilation, Spotless, Scalastyle, and diff checks also passed. JVM execution used the verified native artifact for the exact new base, whose native inputs are unchanged by this PR. Updated the description and requested Spark SQL 4.1 CI coverage with `run-spark-4.1-tests`; hosted checks are pending. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
