morrySnow opened a new pull request, #67889: URL: https://github.com/apache/doris/pull/67889
## Problem A project or filter expression above an outer join could be pushed into the null-generating child even when the expression turns a NULL input into a non-NULL value. For unmatched rows, evaluating that expression below the join lets the later null extension overwrite its computed value and returns an incorrect result. ## Root cause The PushDownProject helper checked only whether one child contained all input slots. It did not account for the different evaluation order on a nullable outer-join side: expressions above the join see NULL-extended rows, while expressions pushed into the child run before null extension. NULL-input reasoning was also duplicated across not-null inference and hypergraph conflict rules. Those implementations did not expose a common fail-closed classification for incomplete or failed constant evaluation. ## Reproduction Use a left table with rows 1 and 2 and a right table with only row 1, then evaluate element_at(coalesce(right_array, [99]), 1) above a left outer join. Before this change, enabling the project-pushdown rule returned NULL for the unmatched row 2, while disabling the rule returned 99. After this change, both plans return 99. ## Fix - Add a shared typed-NULL evaluator with explicit NULL, FALSE, TRUE, OTHER_NON_NULL, and UNKNOWN results. It replaces the exact requested slots with NULL literals of their SQL types and accepts only a fully folded literal; incomplete folding and exceptions become UNKNOWN. - Reuse that evaluator in the two not-null inference semantics and hypergraph conflict-rule generation, preserving each callers accepted result classes. - Guard Project and Filter pushdown through a LogicalJoin nullable side. After child-output containment succeeds, push only when evaluating the complete candidate with all of its input slots set to typed NULL proves an SQL NULL result. - Keep preserved-side pushdown unchanged, handle left, right, full, and ASOF outer joins symmetrically, and leave ON-conjunct pushdown unchanged because ON is evaluated before null extension. Runtime-filter pushdown was audited but is intentionally unchanged here. Although its callers can propagate a CascadesContext, directly using the current evaluator from physical post-processing would also import backend-folding RPC and performance behavior. A future consolidation should first define an FE-only, side-effect-free evaluation policy for that lifecycle. ## Tests - NullInputEvaluatorTest: 3 tests passed, covering all five classifications, typed NullSafeEqual behavior, residual slots, exceptions, and debug-skip folding. - PushDownProjectTest: 9 tests passed, covering Project and Filter on left, right, full, and ASOF outer joins, preserved sides, nested safe expressions, non-NULL constants, and the unchanged ON path. - Not-null inference tests: InferFilterNotNullTest passed 4 tests; aggregate, join, and mark-slot suites passed 28 tests together. - Regression suite outer_join_null_extension passed with 1 suite and 0 failures. It covers the failing expression, mirrored and full outer joins, filters, safe controls, and an explicit rule-disabled comparison. - DISABLE_BUILD_UI=ON ./build.sh --fe completed successfully across 80 modules with zero Checkstyle violations. The host provides Node.js 16 while the current UI toolchain requires Node.js 20 or 22, so the UI stage was excluded. - git diff --check completed with no output. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
