englefly commented on PR #69817:
URL: https://github.com/apache/doris/pull/69817#issuecomment-6568357077
Addressed the `[P2] Preserve key-stat validity before estimating OR
overlaps` comment in 9d237022f67.
I took the second option from the review ("or on OR children without
rescaling again"):
- `visitOr` now calls `enforceValid()` on the left child before it estimates
the overlap, and on each right child. The null counts were already scaled by
the children, so only the validity is repaired here — no second rescaling.
- The merged key statistics of the union are also repaired after the range
union is applied, so a following child of the same OR works on valid statistics.
- The final `enforceValid()` in `FilterEstimation.estimate` stays as it is;
it runs after the union row count is computed and cannot repair it.
Verification:
- `(a = b) or (a <=> b)` with 1000 rows and ndv 1000 now estimates 1 row
(was 1.999, the overlap was `1/1000` because the left branch kept ndv 1000 for
its one output row).
- `(a < b) or (a = 5)` estimates 500 rows (was 500.5 for the same reason).
- Added `FilterEstimationTest#testOrEqualAndNullSafeEqualOverlap` and
`#testOrColumnComparisonUsesValidKeyStatistics`.
- `FilterEstimationTest` (71), `StatisticsTest` (3),
`ExpressionEstimationTest` (17), `JoinEstimateTest` (3), `StatsCalculatorTest`
(20) and `MemoStatsAndCostRecomputerTest` (12) pass, checkstyle is green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
