[
https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Weise updated HIVE-2467:
-------------------------------
Attachment: HIVE-2467.patch
Attached is the initial patch (with unit test coverage). Limited end to end
testing has been done to validate the token sharing across multiple servers. No
scalability testing has been done yet.
Changes all go into the Hadoop 20S shim module (depend on secure Hadoop).
Everything except the ZooKeeper token store implementation logically extends
Hadoop security. AbstractDelegationTokenSecretManager was not designed for
extensibility and this reflects in the subclass introduced here, which
nevertheless duplicates as little code as possible. Longer term Hadoop should
support the token store abstraction or at least more appropriate hooks for
extension and dependency injection, which will allow for significant enhanced
simplicity and clarity (and of course sharing with other projects).
The ZooKeeper backed token store: The delegation keys and tokens are stored in
ZooKeeper. Multiple metastore servers access ZooKeeper and load keys and tokens
from there. Keys are cached and each metastore server instance creates its
dedicated key, shared through ZooKeeper with other server instances for token
validation. Tokens are not cached. Each server will fetch the token from
ZooKeeper when the client establishes a metastore connection (not per request,
but per connection).
By default, none of the extensions introduced with this patch will be used
(previous behavior and dependencies remain intact). To enable ZooKeeper token
store, additional configuration is needed (on top of secure metastore setup):
<property>
<name>hive.cluster.delegation.token.store.class</name>
<value>org.apache.hadoop.hive.thrift.ZooKeeperTokenStore</value>
<description>The delegation token store implementation class.</description>
</property>
<property>
<name>hive.cluster.delegation.token.store.zookeeper.connectString</name>
<value>localhost:2181</value>
<description>The ZooKeeper token store connect string.</description>
</property>
<property>
<name>hive.cluster.delegation.token.store.zookeeper.rootNode</name>
<value>/hcat/tokenstore</value>
<description>The root path for token store data.</description>
</property>
There are two potential enhancements I would like to put up for discussion:
1) Partition token storage by master key id (right now all tokens are children
of a single ZooKeeper node. This will make it easier to manually inspect the
store and make token expiration more efficient (no need to read all children at
once).
2) Introduce dependency injection point for
HadoopThriftAuthBridge20S.createServer, which can be used to plug-in
alternative implementations of TokenStore, DelegationTokenManager and
potentially other objects for customization and unit testing.
> HA Support for Metastore Server
> --------------------------------
>
> Key: HIVE-2467
> URL: https://issues.apache.org/jira/browse/HIVE-2467
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.8.0
> Reporter: Thomas Weise
> Attachments: HIVE-2467.patch
>
>
> We require HA deployment for metastore server for HCatalog:
> * Multiple server instances run behind VIP
> * Database provides HA
> Metastore server instances will need to be able to share any state required
> for VIP outside RDBMS. As of Hive 0.8 affected conversational state that
> needs to support VIP/HA setup is limited to current delegation tokens. Is
> this correct?
> We are planning to use ZooKeeper to share current delegation tokens and
> master keys between nodes of the VIP. ZK is already (optionally) used by Hive
> for concurrency control. Access to ZK would be limited on the network level
> or in the future, when ZooKeeper supports security, through Kerberos, similar
> to NN access.
> Currently Hive taps into Hadoop core security delegation token support
> through extension of
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager<TokenIdent>
> A solution could amend the Hive specific extension to support:
> * Pluggable delegation token and master key store (ZooKeeper as alternative
> for in-memory AbstractDelegationTokenSecretManager)
> * Delegation token retrieval from token store when not found in memory
> (wrap/extend retrievePassword(...))
> * Cancellation of token in token store
> * Purging of expired tokens from token store
> http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
