[
https://issues.apache.org/jira/browse/HIVE-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190035#comment-13190035
]
Phabricator commented on HIVE-2720:
-----------------------------------
enis has commented on the revision "HIVE-2720 [jira] Merge MetaStoreListener
and HiveMetaHook interfaces".
The patch as it is, does not change the event listener calling semantics or
hivemetahook calling semantics. For something like an audit log, the order for
the registered hooks become really important, and it should indeed be the first
registered hook. That was one of the reasons that the
HiveMetaStore.getDefaultHooks() precede all the user-defined hooks.
As you suggested we can add transactional semantics to the hook interface,
but I guess that would require exposing startTransaction, rollback and commit
interfaces, and implementing something like a two phase commit to make sure to
keep consistency with the external data. Without it, I don't think we can
implement correct behavior.
Having said that, I am very reluctant to go that way until some real need
arises. So what I propose we can do is:
- Keep the HiveMetaHook as it is in trunk and HiveMetaStoreClient.
- introduce HiveExtendedMetaHook interface to limited private to replace
HiveMetaStoreEventListener, and allow only hive and hcat implementations (by
documenting it). We guarantee correct behavior by carefully ordering the hooks.
- Keep the current semantics as in the patch, but improve on the javadoc for
the method semantics.
Would that work, what do you think?
INLINE COMMENTS
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java:105
Renaming methods from onXXX -> postXX makes sense.
The MetaStoreEventListener's can interact with the function execution by
throwing exceptions, or can act differently on the outcome of the method call
(like the HbaseStorageHandler). We can rename MetaStoreEventListener to
MetaStoreHook, or else, we can re-purpose HiveMetaHook, and deprecate
MetaStoreEventListener instead. Wdyt, any other suggestions?
I'll update the patch shortly.
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java:199
I'll update the patch to add all the method hooks.
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaHook.java:66 The
patch as it is, does not change the event listener calling semantics or
hivemetahook calling semantics. For something like an audit log, the order for
the registered hooks become really important, and it should indeed be the first
registered hook. That was one of the reasons that the
HiveMetaStore.getDefaultHooks() precede all the user-defined hooks.
As you suggested we can add transactional semantics to the hook interface,
but I guess that would require exposing startTransaction, rollback and commit
interfaces, and implementing something like a two phase commit to make sure to
keep consistency with the external data. Without it, I don't think we can
implement correct behavior.
Having said that, I am very reluctant to go that way until some real need
arises. So what I propose we can do is:
- Keep the HiveMetaHook as it is in trunk and HiveMetaStoreClient.
- introduce HiveExtendedMetaHook interface to limited private to replace
HiveMetaStoreEventListener, and allow only hive and hcat implementations (by
documenting it). We guarantee correct behavior by carefully ordering the hooks.
- Keep the current semantics as in the patch, but improve on the javadoc for
the method semantics.
Would that work, what do you think?
REVISION DETAIL
https://reviews.facebook.net/D1299
> Merge MetaStoreListener and HiveMetaHook interfaces
> ---------------------------------------------------
>
> Key: HIVE-2720
> URL: https://issues.apache.org/jira/browse/HIVE-2720
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC, Metastore, ODBC, Security
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Attachments: HIVE-2720.D1299.1.patch, HIVE-2720.D1299.2.patch,
> HIVE-2720.D1299.3.patch
>
>
> MetaStoreListener and HiveMetaHook both serve as a notification mechanism for
> metastore-related events. The former is used by hcat and the latter is by the
> hbase-storage handler, and invoked by the client.
> I propose to merge these interfaces, and extend the MetaStoreListener, to add
> most of the on- and pre- methods at the Thrift interface. This way, extending
> metastore will be easier, and validation, storage-driver notification, and
> enforcement can be delegated to individual listeners. Besides, more
> functionality can be plugged-in by Hcat at this level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
