[
https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665644#comment-15665644
]
Ruslan Dautkhanov commented on SPARK-17076:
-------------------------------------------
HIVE-13076 added FK constraint to Hive / HMS.
Joining two tables without declared relation can produce and number of rows.
On one extreme case, joining two tables produces zero rows (if there is no
overlaps in those join set of ids).
On another extream case, joining two tables will produce N x M rows where N and
M are number of rows in each case.
Luckliy, most often joining happens using columns that can have declared
constraints on them.
For example,
customers.customer_id = purchases.customer_id AND purchases.product_id =
products.product_id
If you'd build on HIVE-13076, Spark CBO will know exactly that
customers.customer_id = purchases.customer_id AND purchases.product_id =
products.product_id will always produce 1 row for each join (join cardinality =
# of rows on the biggest table).
> Cardinality estimation of join operator
> ---------------------------------------
>
> Key: SPARK-17076
> URL: https://issues.apache.org/jira/browse/SPARK-17076
> Project: Spark
> Issue Type: Sub-task
> Components: Optimizer
> Affects Versions: 2.0.0
> Reporter: Ron Hu
>
> support cardinality estimates for equi-join, Cartesian product join, and
> outer join, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
