[
https://issues.apache.org/jira/browse/HIVE-18940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405643#comment-16405643
]
Thejas M Nair commented on HIVE-18940:
--------------------------------------
[~vihangk1]'s approach of accumulating the events and only getting the lock
towards end seems like a reasonable way to reduce the duration lock is held to
a very small time, for metastore calls that lead to several events. Other parts
of the transactions can go ahead in parallel.
It doesn't have to use same commitID for all events, getting new values should
be OK, as long as the lock on NOTIFICATION_SEQUENCE is obtained at the end.
> Hive notifications serialize all write DDL operations
> -----------------------------------------------------
>
> Key: HIVE-18940
> URL: https://issues.apache.org/jira/browse/HIVE-18940
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 3.0.0
> Reporter: Alexander Kolbasov
> Priority: Major
>
> The implementation of DbNotificationListener uses a single row to store
> current notification ID and uses {{SELECT FOR UPDATE}} to lock the row. This
> serializes all write DDL operations which isn't good.
> We should consider using database auto-increment for notification ID instead.
> Especially on mMySQL/innoDb it is supported natively with relatively
> light-weight locking.
> This creates potential issue for consumers though because such IDs may have
> holes. There are two types of holes - transient hole for a transaction which
> have not committed yet and will be committed shortly and permanent holes for
> transactions that fail. Consumers need to deal with it. It may be useful to
> add DB-generated timestamp as well to assist in recovery from holes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
