Baymine opened a new pull request, #65900:
URL: https://github.com/apache/doris/pull/65900
### What problem does this PR solve?
Issue Number: close #65899
Problem Summary:
When an Olap or external (Hive/Iceberg/etc.) table has not been ANALYZE'd,
its row count is propagated as UNKNOWN_ROW_COUNT (-1) from BE/catalog.
`StatsCalculator` then clamps this -1 up to 1 row so the cost model (which
cannot tolerate negatives) keeps working. The resulting "1 row" looks tiny, so
`JoinUtils.checkBroadcastJoinStats` happily picks that table as the broadcast
build side — even though its true size is unknown and potentially huge. At
runtime the un-ANALYZE'd large table gets fully materialized on every BE node,
causing OOM.
This adds a `rowCountWasUnknown` flag to `Statistics` (propagated through
the copy ctor, the 6-arg ctor, `StatisticsBuilder`, and the `withRowCount*`
family) that records whether the row count was clamped from UNKNOWN.
`StatsCalculator` sets it in `computeOlapScan` and `computeCatalogRelation`
(refreshing it in the external path so a row count recovered from per-column
stats clears the flag). `JoinUtils.checkBroadcastJoinStats` now refuses
broadcast when the flag is set, in addition to the existing guards.
### Release note
When a table has not been analyzed, its row count is unknown and is
internally clamped to 1. Previously such a table could be chosen as the
broadcast join build side because 1 row looks small, leading to OOM at runtime
on large un-analyzed tables. The optimizer now refuses broadcast for a build
side whose row count was clamped from unknown, falling back to shuffle join
instead.
### Check List (For Author)
- Test
- [x] Unit Test
- Behavior changed:
- [x] Yes. An un-analyzed table whose row count was previously clamped
to 1 will no longer be eligible as a broadcast join build side; the optimizer
falls back to shuffle join. This avoids OOM but may change plans for queries
that previously (possibly wrongly) used broadcast.
- Does this need documentation?
- [x] No. Internal optimizer stats-tracking change; no user-facing knob.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
