andygrove opened a new issue, #2185:
URL: https://github.com/apache/datafusion-ballista/issues/2185
## Describe the bug
With adaptive execution enabled (`ballista.planner.adaptive.enabled=true`)
at `target_partitions=1`, TPC-DS q61 returns an empty result where
single-process DataFusion returns one row:
```
Query 61 VERIFY MISMATCH: Execution error: result mismatch: expected 1 rows,
got 0 rows
```
Surfaced by extending the TPC-DS SF1 correctness gate to cover the adaptive
planner and single-partition execution (#2182).
## Scope
- Only reproduces with **AQE on**. The same query at `--partitions 1` with
the static planner passes.
- Does not reproduce at `--partitions 16`.
- q61 contains no `UNION`, so this appears distinct from #2184 (which covers
the UNION scan-restriction failures at 1 partition). Filing separately on that
basis; if root-causing shows a shared cause they can be merged.
## To Reproduce
Generate TPC-DS SF1 Parquet with `tpcgen-cli`, start a scheduler and
executor, then:
```
tpcds --host localhost --port 50050 --path <data> --partitions 1 --verify \
--query 61 \
-c datafusion.optimizer.prefer_hash_join=false \
-c ballista.planner.adaptive.enabled=true
# -> Query 61 VERIFY MISMATCH: expected 1 rows, got 0 rows
```
Passes with `--partitions 16`, and with `--partitions 1` when the adaptive
planner is off.
## Expected behavior
The adaptive planner should return the same single row as single-process
DataFusion, at every partition count.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
