[
https://issues.apache.org/jira/browse/HIVE-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13652345#comment-13652345
]
Jitendra Nath Pandey commented on HIVE-4233:
--------------------------------------------
bq. Since the kerberos expiry is usually in hours or days, having a simple
logic of doing relogin if the previous login was several minutes back should
fix this issue.
That sounds right, UGI.reloginFromKeytab already does it, checks if sufficient
time has elapsed. The 'sufficient time' can be configured. This method also
checks for TGT refresh time, therefore just calling reloginFromKeytab before
every connection should be a reasonable fix.
> The TGT gotten from class 'CLIService' should be renewed on time
> -----------------------------------------------------------------
>
> Key: HIVE-4233
> URL: https://issues.apache.org/jira/browse/HIVE-4233
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 0.10.0
> Environment: CentOS release 6.3 (Final)
> jdk1.6.0_31
> HiveServer2 0.10.0-cdh4.2.0
> Kerberos Security
> Reporter: Dongyong Wang
> Priority: Critical
> Attachments: 0001-FIX-HIVE-4233.patch
>
>
> When the HIveServer2 have started more than 7 days, I use beeline shell to
> connect the HiveServer2,all operation failed.
> The log of HiveServer2 shows it was caused by the Kerberos auth failure,the
> exception stack trace is:
> 2013-03-26 11:55:20,932 ERROR hive.ql.metadata.Hive:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1084)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:51)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:61)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2140)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2151)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getDelegationToken(Hive.java:2275)
> at
> org.apache.hive.service.cli.CLIService.getDelegationTokenFromMetaStore(CLIService.java:358)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:127)
> at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1073)
> at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1058)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:565)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor52.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1082)
> ... 16 more
> Caused by: java.lang.IllegalStateException: This ticket is no longer valid
> at
> javax.security.auth.kerberos.KerberosTicket.toString(KerberosTicket.java:601)
> at java.lang.String.valueOf(String.java:2826)
> at java.lang.StringBuilder.append(StringBuilder.java:115)
> at
> sun.security.jgss.krb5.SubjectComber.findAux(SubjectComber.java:120)
> at sun.security.jgss.krb5.SubjectComber.find(SubjectComber.java:41)
> at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:130)
> at
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:328)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:325)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:128)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:106)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:172)
> at
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:209)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:195)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:162)
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:175)
> at
> org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:277)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:163)
> ... 20 more
> I check the code of HiveAuthFactory.loginFromKeytab,it does not schedule a
> timer to renew the TGT. So I suspect this is the reason of the kerberos auth
> failure?
> Thanks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
