lxc512157407 commented on PR #24821: URL: https://github.com/apache/datafusion/pull/24821#issuecomment-5649977307
**Benchmark results** (4M rows, MemTable, nullable column with zero NULLs) | Query | main (ms) | PR (ms) | Delta | |-------|-----------|----------|-------| | sum(id) no filter | 0.679 | 0.667 | -0.012 | | sum(id) WHERE id IS NOT NULL | **0.873** | **0.743** | **-0.130 (~15% faster)** | The optimization reduces the per-batch overhead of FilterExec (stream wrapping, coalescer, metrics) — measurable on `sum()` where the scan itself dominates. The `count(*)` case shows noise in the opposite direction and is not indicative. This confirms the optimization has a measurable benefit on workloads with nullable join keys that are provably non-null in practice (the overwhelmingly common case for PK/FK relationships). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
