lxc512157407 commented on issue #24942: URL: https://github.com/apache/datafusion/issues/24942#issuecomment-5601097657
Hi @2010YOUU01 - here's a minimal SQL reproducer that triggers this: ```sql -- Setup: two tables with nullable join keys CREATE TABLE a (id INT, val INT); INSERT INTO a VALUES (1, 10), (2, 20), (3, 30); CREATE TABLE b (id INT, val INT); INSERT INTO b VALUES (1, 100), (2, 200); -- Join planning inserts IS NOT NULL on nullable keys (filter_null_join_keys) -- EXPLAIN shows the filter is pushed down to the scan: SELECT a.val FROM a JOIN b ON a.id = b.id; -- Even when data provably contains no NULLs (primary keys), -- the filter still runs at 100% selectivity: SELECT a.val FROM a WHERE a.id IS NOT NULL; -- With stats that report null_count=0 on id column: -- the IS NOT NULL conjunct is provably true and could be dropped. ``` The PR #24821 adds statistics-driven simplification that drops these provably-true IS NOT NULL conjuncts at FilterExec build time. The issue tracks adding this optimization to the join planning phase as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
