[
https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248115#comment-13248115
]
alex gemini commented on HIVE-1721:
-----------------------------------
The original thought is increase the map side small table size,this is very
depend on how we chunk big table.if big table is chunk into buckets for 16
buckets,the small table must be automatic change to 16 buckets too(same logic
bucket map join).if big table is partition by (region string),the small table
also need to partition by (region) first and also make sure the smallest chunk
size is not bigger than current small table size define by hive.The partition
case is more often,we can avoid common join by always let small table chunk
like big table's format.
> use bloom filters to improve the performance of joins
> -----------------------------------------------------
>
> Key: HIVE-1721
> URL: https://issues.apache.org/jira/browse/HIVE-1721
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Namit Jain
> Labels: gsoc, gsoc2012, optimization
>
> In case of map-joins, it is likely that the big table will not find many
> matching rows from the small table.
> Currently, we perform a hash-map lookup for every row in the big table, which
> can be pretty expensive.
> It might be useful to try out a bloom-filter containing all the elements in
> the small table.
> Each element from the big table is first searched in the bloom filter, and
> only in case of a positive match,
> the small table hash table is explored.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
