shihchengwei-lab commented on PR #27597:
URL: https://github.com/apache/datafusion/pull/27597#issuecomment-7036916326
I reproduced an `ORDER BY` regression on `a2b0c6e4`. With the same two-row
CSV and one partition, the base commit (`c6c1cc31`) keeps `SortExec` and
returns `t = 1, 2`. The PR removes `SortExec` and returns `t = 2, 1`.
`reorder.csv` has no header. The empty `b` field reads as `NULL`.
```csv
1,,0,2
1,,0,1
```
```sql
CREATE EXTERNAL TABLE reorder (
a INT NOT NULL, b INT, c INT NOT NULL, v INT NOT NULL,
UNIQUE (a, b)
)
STORED AS CSV
WITH ORDER (a ASC, b ASC, c ASC)
OPTIONS ('format.has_header' 'false')
LOCATION 'reorder.csv';
SELECT a AS x, v AS t, b AS y, c AS z
FROM reorder
ORDER BY x, y, z, t;
```
The patch correctly remaps `UNIQUE (a, b)` to `(x, y)`. That exposes an
existing assumption in the ordering check: it accepts nullable `y` because `y`
is in the middle of `(x, y, z)`. Both `(1, NULL)` keys are valid, and the rows
tie on all three sort columns, so `t` must break the tie. Could we require a
non-null guarantee for every component of a `UNIQUE` key before using it to
skip the rest of the sort?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]