zhuqi-lucas opened a new issue, #27053: URL: https://github.com/apache/datafusion/issues/27053
### Is your feature request related to a problem or challenge? When `preserve_order=true`, each output partition of `RepartitionExec` runs a `StreamingMergeBuilder` over all of its input partitions, the same merge that `SortPreservingMergeExec` uses. It has the behavior described in #27003: when the inputs' key ranges are disjoint, the merge pulls from one input at a time. Neither EXPLAIN nor EXPLAIN ANALYZE makes this visible. The plan line is ``` RepartitionExec: partitioning=..., input_partitions=N, preserve_order=true, sort_exprs=... ``` and the metrics are only the baseline ones (`elapsed_compute`, `output_rows`). A plan with no `SortPreservingMergeExec` in it can still be bounded by a serial merge, and the only signal is low CPU usage while the query runs. ### Describe the solution you'd like 1. Make the merge visible in the plan display, e.g. `preserve_order=true (merges N inputs per output partition)`. 2. Add merge metrics to the shared streaming merge, for example time spent waiting on inputs. `RepartitionExec` (preserve-order path) and `SortPreservingMergeExec` both build on it, so a merge that reads its inputs serially would show up in EXPLAIN ANALYZE for either. ### Describe alternatives you've considered Documenting that `preserve_order=true` implies a per-partition merge, without changing display or metrics. ### Additional context Related: #27003. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
