andygrove commented on issue #6399: URL: https://github.com/apache/datafusion-comet/issues/6399#issuecomment-5897037315
Phase 3, expressions and casts: all 12 PRs have been reviewed against 1.0.0 and against Spark. Every reproducer was then run on 1.0.0 and 1.1.0-rc1 builds, which covers Phase 4 for this area. The #6261 check is done too: that hang was already in 1.0.0. Five regressions that ship in 1.1.0 are confirmed, and #6402 has the details: - #6423, from #4775: `regr_slope`, `regr_intercept` and `regr_r2` return large wrong values instead of NULL for a constant variable merged across partitions. - #6424, from #5692: a typed `Dataset.filter` over a native aggregate returns wrong rows. #6339 fixes it on `main` and needs a backport. - #6425, from #5692: decimal-returning DSv2 functions lose their scale in the codegen dispatcher, e.g. 0.03 instead of 3.00. - #6426, from #5638 and #5773: the native Iceberg `years`, `months`, `days` and `hours` differ from Iceberg for rare pre-1970 timestamps. - #6334, from #5452: a native `IF` with a map or struct literal branch throws when every row of a batch takes the same branch. Each one has a draft fix PR open or in progress, linked from its issue. No regression was found in #5472, #5166, #5415, #5614, #5280 or #5682. Several of them fix wrong answers that 1.0.0 gave. #4816 changes the order of values in `listagg` only where Spark leaves that order unspecified, so it's not a regression. Why review missed these: - A path became native by default without a test for the case where it diverges from Spark: a constant merged from several partitions in #4775, pre-1970 unit boundaries in #5638, and a batch where every row takes one branch in #5452. - #5692's catch-all sent new shapes through the codegen dispatcher that the dispatcher's tests never covered: sliced boolean inputs and decimal results whose scale differs from the declared type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
