[
https://issues.apache.org/jira/browse/SPARK-20678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhenhua Wang updated SPARK-20678:
---------------------------------
Description: In filter estimation, we update column stats for those columns
in filter condition. However, if the number of rows decreases after the filter
(i.e. the overall selectivity is less than 1), we need to update (scale down)
the number of distinct values (NDV) for all columns, no matter they are in
filter conditions or not.
> Ndv for columns not in filter condition should also be updated
> --------------------------------------------------------------
>
> Key: SPARK-20678
> URL: https://issues.apache.org/jira/browse/SPARK-20678
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Zhenhua Wang
>
> In filter estimation, we update column stats for those columns in filter
> condition. However, if the number of rows decreases after the filter (i.e.
> the overall selectivity is less than 1), we need to update (scale down) the
> number of distinct values (NDV) for all columns, no matter they are in filter
> conditions or not.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
