shihchengwei-lab commented on PR #27597:
URL: https://github.com/apache/datafusion/pull/27597#issuecomment-7036916326

   I reproduced an `ORDER BY` regression on `a2b0c6e4`. With the same two-row 
CSV and one partition, the base commit (`c6c1cc31`) keeps `SortExec` and 
returns `t = 1, 2`. The PR removes `SortExec` and returns `t = 2, 1`.
   
   `reorder.csv` has no header. The empty `b` field reads as `NULL`.
   
   ```csv
   1,,0,2
   1,,0,1
   ```
   
   ```sql
   CREATE EXTERNAL TABLE reorder (
     a INT NOT NULL, b INT, c INT NOT NULL, v INT NOT NULL,
     UNIQUE (a, b)
   )
   STORED AS CSV
   WITH ORDER (a ASC, b ASC, c ASC)
   OPTIONS ('format.has_header' 'false')
   LOCATION 'reorder.csv';
   
   SELECT a AS x, v AS t, b AS y, c AS z
   FROM reorder
   ORDER BY x, y, z, t;
   ```
   
   The patch correctly remaps `UNIQUE (a, b)` to `(x, y)`. That exposes an 
existing assumption in the ordering check: it accepts nullable `y` because `y` 
is in the middle of `(x, y, z)`. Both `(1, NULL)` keys are valid, and the rows 
tie on all three sort columns, so `t` must break the tie. Could we require a 
non-null guarantee for every component of a `UNIQUE` key before using it to 
skip the rest of the sort?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to