andygrove opened a new issue, #7204:
URL: https://github.com/apache/datafusion-comet/issues/7204
## What / Why
This epic tracks functional regressions in 1.2.0 against the 1.1.0 release.
They come from an audit of `branch-1.2` (head 889856dc5) against the `1.1.0`
tag. I'll keep this description updated as fixes and backports land.
A regression here is something that worked in 1.1.0 with default configs and
is broken or worse in 1.2.0. That covers a wrong or different result, a new
failure, a loud failure that became a silent wrong answer, or a severe slowdown
on a default path. Bugs that 1.1.0 already had are not listed. Neither are bugs
in features that are off by default.
Each issue below carries the `regression` label.
## Regressions on `branch-1.2`
- [ ] #7118: invalid UTF-8 bytes are replaced with U+FFFD in two paths that
1.1.0 kept on Spark. One is a Text-format broadcast build side, converted by
#6786 (`spark.comet.convert.broadcastBuildSide.enabled`, on by default). The
other is `min`/`max` over a string cast from binary, now native through #6927.
The fix for the build side is #7123, which keeps Text build sides on Spark by
default. It is merged to `main` but **not yet on `branch-1.2`**, so it needs a
backport. The `min`/`max` case falls under the ordering difference the
compatibility guide already documents.
- [ ] #5022: #4974 made RANGE frames with an explicit offset on a DATE
`ORDER BY` native; 1.1.0 fell back for any such frame. The overflow guard only
falls back above 2,144,550,751 days. An offset whose bound overflows Date32
lets a NULL peer group into the frame. With an offset of 100000000 days, Spark
gives `2,2,1,2` and 1.2.0 gives `2,4,3,2`. The fix is to lower the threshold to
about 92M days. The issue is on the 1.3.0 milestone and should move to 1.2.0.
- [ ] #6880: #6867 runs a native `PartialMerge` as `Single`, which
DataFusion 55.2 cannot spill when the input is sorted on a prefix of the
grouping keys. A `count(DISTINCT k)` over input sorted on the group key now
fails the task. In the audit it failed at every pool size from 2 MiB to 256
MiB, with `Additional allocation failed for GroupedHashAggregateStream`. 1.1.0
returned the right count there when keys were unique, and a wrong count when
keys repeated (#6857).
- [ ] #6936: #6927 made string and binary `min`/`max` native, where 1.1.0
fell back. DataFusion's `MinMaxBytesAccumulator` under-reports its memory, so
these aggregates spill late and can run out of memory. Results are not affected.
- [ ] #7202: #6059 resolves ABFS authentication through Hadoop on the
driver. When the driver declines an auth type (`Custom`,
`UserboundSASWithOAuth`, custom SAS providers, RefreshToken and UserPassword
providers), the scan still plans natively and every task fails. The same
happens when the driver cannot reflect into hadoop-azure, which also hits users
who configure Azure only through environment variables. 1.1.0 read these
natively through the ambient identity. This was found by reading the code.
- [ ] #7203: #4727 made `collect_set` native in `count(DISTINCT ...)`
chains. On Spark before 4.2, the documented `collect_set` floating-point
difference reaches that shape: -0.0 and 0.0 stay as two entries, and NaNs merge
into one. 1.1.0 kept the shape in Spark and matched it.
## Behavior changes for the release notes
These are intended, but a user upgrading from 1.1.0 can see them.
- #6494 renamed `spark.comet.parquet.write.enabled` and
`spark.comet.iceberg.write.enabled` to `spark.comet.write.parquet.enabled` and
`spark.comet.write.iceberg.enabled`, with no alias. Users who opted in on 1.1.0
under the old names now silently get Spark's writers.
- #6447: on a Parquet DOUBLE or FLOAT column with a bloom filter, `d = 0.0`,
`d <=> 0.0` and `d IN (0.0, ...)` against a file that holds only `-0.0` now
return the row. Spark's own pushdown, and 1.1.0, skip the file and return
nothing. The 1.2.0 answer is the correct one, and
`compatibility/floating-point.md` documents it.
- #5420: under ANSI, `AVG(DECIMAL(38,0))` overflow now raises
`NUMERIC_VALUE_OUT_OF_RANGE` where 1.1.0 returned NULL. This matches Spark.
- #7016 shares one decoded broadcast per executor, through a new value
class. `CometKryoRegistrator` registers it. Only a user who lists Comet's
classes by hand under `spark.kryo.registrationRequired` needs to add it.
## Audit scope
- **Baseline:** every fix in 1.1.0 has its original on `branch-1.2`. That
includes the four PRs merged straight into `branch-1.1`: #6487, #6511, #6488
and #6526.
- **Dependencies:** DataFusion moved from 55.1.0 to 55.2.0 (#6766). arrow
and parquet stay at 59.3.0. iceberg-rust moved from `bb1e4a4` to `1f3bc34`
(#6649, #6851), and both ranges were reviewed.
- **Phase 1:** I triaged 251 production PRs from the 388 commits in
`36ab57c68..branch-1.2`, reading the code. 3 rated HIGH and 31 MEDIUM.
- **Phase 2:** every HIGH and MEDIUM PR was checked against Spark with
targeted tests on the Spark 4.1 profile. So were the 14 PRs from an earlier
pass on 2026-10-19: #6943, #6730, #6899, #6927, #4587, #6350, #6076, #6786,
#6725, #4798, #6953, #5496, #4974 and #6851. The PRs that showed no regression
are:
- **Plans and aggregation:** #4565, #6666, #6835, #5420, #6547, #6646,
#6459, #6314.
- **Floating point:** #6457, #6518, #6691, #6599, #6447, #6968, #6563.
- **Expressions:** #5130, #5956, #5403, #6716, #6680, #6180, #4715.
- **Shuffle and spill:** #6824, #7065, #6548, #6698.
- **Scans and object stores:** #7094, #6990, #6753.
- **Memory and broadcast:** #6261, #7016.
- **Iceberg reads:** #7132, #6106, #6649.
- **Phase 3:** I classified the 73 open bugs filed since the 1.1 branch cut.
Only #6880 and #6936 are 1.2 regressions. #6256 (JVM columnar shuffle with
`spark.shuffle.checksum.enabled=false`) looks fixed on `branch-1.2` by #7065.
- **Not covered:** Spark profiles other than 4.1 were mostly not run, and
nothing ran against real cloud object stores or Celeborn.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
