[
https://issues.apache.org/jira/browse/HIVE-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099716#comment-18099716
]
Denys Kuzmenko commented on HIVE-29762:
---------------------------------------
cc [~abstractdog] yet another concurrent add_partition
> Synchronization logic added as a part of HIVE-24428 is having hash collision
> issue.
> -----------------------------------------------------------------------------------
>
> Key: HIVE-29762
> URL: https://issues.apache.org/jira/browse/HIVE-29762
> Project: Hive
> Issue Type: Bug
> Reporter: Dayakar M
> Assignee: Dayakar M
> Priority: Major
> Labels: pull-request-available
>
> Synchronization logic added as a part of HIVE-24428 is having hash collision
> issue.
> To prevent concurrent issues as a part of _add_partitions_ request, added
> _com.google.common.util.concurrent.Striped.lock(stripes)_ mechanism which
> internally creates lock objects by hashing the given key, here it also uses
> the stripes value while calculating the hash. Sometimes for unrelated keys it
> is generating the same hash which resulting same lock object.
> The intent of this locking mechanism is to prevent concurrent requests for
> the add_partitions request for the same table but due to hash collision for
> unrelated keys(db.table names) it is using same lock object which is
> preventing other unrelated add_partition requests.
> In this case if stripe size increased to some bigger value like 1024 then it
> is generating different hash which results different lock keys but there
> could be still hash collision can happen and increasing stripe size will
> increase memory overhead.
> So this has to be checked and need a proper fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
