> On Sept. 5, 2017, 7:37 p.m., Janaki Lahorani wrote: > > shims/common/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java > > Line 155 (original), 155 (patched) > > <https://reviews.apache.org/r/62092/diff/1/?file=1815840#file1815840line155> > > > > Why do we need this change?
When you use Hive.get() instead of Hive.get(conf, <class>) you use sessionConf to instantiate HMSClient. This is needed when impersonation is turned on. When it is on, HiveSessionImpleWithUGI stores a marker in the sessionConf with the key "hive.metastore.token.signature". This value is fetched when we instantiate HMSClient to check if we should use Digest-MD5 or Kerberos to authenticate the client after opening the connection. Currently, implementation uses server config to instantiate the HMSClient which doesn't have this value set, so it will always try to use Kerberos when impersonation is turned on. > On Sept. 5, 2017, 7:37 p.m., Janaki Lahorani wrote: > > shims/common/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java > > Line 102 (original), 102 (patched) > > <https://reviews.apache.org/r/62092/diff/1/?file=1815841#file1815841line102> > > > > Can ownerStr be null? If so, does it make sense to get the current > > user from ugi? I don't think ownerStr can ever be null. I can add a check here and throw if it is null. We cannot use current user from the ugi because that would be the superuser (oozie) and the token owner will become oozie. In such a case, I think the end user will not be able to use the token. > On Sept. 5, 2017, 7:37 p.m., Janaki Lahorani wrote: > > shims/common/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java > > Line 108 (original), 108 (patched) > > <https://reviews.apache.org/r/62092/diff/1/?file=1815841#file1815841line108> > > > > Just curious: in the case of systest, oozie, and hive who is owner, > > real user and renewer? Assuming you mean systest is the end-user, it depends on whether impersonation is ON/Off and what is passed by Oozie in the GetDelegationReq. The owner and renewer comes in from the GetDelegationReq and the realUser is Oozie. Thanks for asking this question because I realized that the real user changed from previous implementation after this patch. I will update the patch so that previous behaviour is maintained. > On Sept. 5, 2017, 7:37 p.m., Janaki Lahorani wrote: > > shims/common/src/main/java/org/apache/hadoop/hive/thrift/HiveDelegationTokenManager.java > > Lines 119-124 (original), 119-124 (patched) > > <https://reviews.apache.org/r/62092/diff/1/?file=1815842#file1815842line119> > > > > IMHO the reasons behind always using hive here are as follows: > > 1. Currently the intermediate app (oozie) proxying as enduser > > (systest) is based on authorization only through ProxyUsers. The identity > > of systest is not authenticated to HS2. HS2 trusts that oozie knows > > systest. > > 2. When HS2 connects to HMS to save delegation token, HS2 cannot > > connect as systest or oozie using kerberos because HS2 doesn't have access > > to the keytab files of systest or oozie. Though if oozie's delegation > > token is already setup, HS2 can potentially use the same. > > 3. If HS2 is generating a delegation token to be saved in HMS, a > > simple security model will be that HS2 always uses its own identity to save > > the token because HS2 generated the token, and HS2 generated the token > > either based on trust or based on authentication. Either way, the entity > > that generated the token is the entity that saves the token. > > > > In the above case, I would like to understand whether the delegation > > token is saved into HMS as hive (after authenticating to HMS as hive using > > kerberos) or as oozie (after authentication to HMS as oozie using oozie's > > delegation token)? > > > > In the future if we are to pick kerberos with proxy, the authentication > > to HMS will be using a proxied ticket instead of hive. > > > > :) I am just putting it all down so that I remember as well. Please > > feel to free to trim it to what you think is appropriate. I will update the comment to reflect some of thoughts you mentioned above. - Vihang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62092/#review184571 ----------------------------------------------------------- On Sept. 5, 2017, 6:53 p.m., Vihang Karajgaonkar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62092/ > ----------------------------------------------------------- > > (Updated Sept. 5, 2017, 6:53 p.m.) > > > Review request for hive, Aihua Xu, Janaki Lahorani, Sergio Pena, and Sahil > Takiar. > > > Bugs: HIVE-17368 > https://issues.apache.org/jira/browse/HIVE-17368 > > > Repository: hive-git > > > Description > ------- > > HIVE-17368 : DBTokenStore fails to connect in Kerberos enabled remote HMS > environment > > > Diffs > ----- > > itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java > bbec37eea76517e9d42e60b26d85cd0b22965cc9 > > itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithDBTokenStore.java > d690aaa673a50785561750f4f461ec867b6f0abc > > itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoopAuthBridge23.java > 36a9ea830a62496351103bde143bc9dd22c9ba23 > itests/util/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java > 71f9640ad217ad60377720489e1ccc71506e51d7 > ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java > ffce1d1aec8728840bb8ef726db1b600a9aeef38 > > service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java > 00a7e742cabd2fc36faa464b29250b5a6a9b1159 > shims/common/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java > d6dc0796e77591d3afca8dbd29c3aa0eff255dd0 > > shims/common/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java > 5299e18743aa45c539287b335f95e8ce8df0fc35 > > shims/common/src/main/java/org/apache/hadoop/hive/thrift/HiveDelegationTokenManager.java > b3e4a7608282be603e79d1d101679e239a5219b0 > > > Diff: https://reviews.apache.org/r/62092/diff/1/ > > > Testing > ------- > > > Thanks, > > Vihang Karajgaonkar > >
