Hello Shakti
The configurations mentioned in the link above are all OK and we are able
to connect to Hive from Kylin and Hive CLI. Even we are able to connect to
meta store in a JAR package but run with classpath setting with
HIVE_CONF_DIR.
In our web applications, what we are doing is like the codes below:
HiveConf conf=new HiveConf();
> File f=new File(ConfUtil.getHiveConfDir()+File.separator+"hive-site.xml");
> if(f.exists()){
> conf.addResource(f.toURI().toURL());
> }
> else{log.error(f.toString()+"nonexist.");}
> try{
> client=new HiveMetaStoreClient(conf);
> }
> catch(Exception e){log.error("HiveMetaStoreClient exeception:
> "+e.getMessage());e.printStackTrace();}
Note: our web application deployed as a WAR package under the Jetty webapps.
Thanks again,
Wenxing
On Thu, Jul 27, 2017 at 8:58 PM, shakti singh Shekhawat <
shaktisingh.shekhawa...@gmail.com> wrote:
> Hi Wenxing,
>
> Some of the changes I can see in hive-site.xml in Kerberized cluster as
> compared to our non-kerberized one is:
> hive.metastore.*sasl.enabled* -->
> *<value>true</value> --This property is false in non-kerberized
> cluster*
> hive.server2.authentication --> <value>KERBEROS</value>
>
> Adding the below links(please refer as per your distribution) for your
> reference for all the properties that are needed to be set in hive-site.xml:
> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/
> bk_security/content/kerb-config-hive-site.html
> https://www.cloudera.com/documentation/enterprise/5-2-
> x/topics/cdh_sg_hive_metastore_security.html
>
> The error you pasted above also points to SASL issue:
> 2017-07-27 10:29:16,873 ERROR
> *org.apache.thrift.transport.**TSaslTransport:SASL
> negotiation failur*e
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided
>
> Please let me know if the above helps in debugging the issue. Also, please
> let us know in case you are able to connect to Hive from an edge node or
> through other tools.
>
> Thanks,
> Shakti
>
>
> On Thu, Jul 27, 2017 at 2:04 AM, wenxing zheng <wenxing.zh...@gmail.com>
> wrote:
>
>> In my web application, I am using the HiveMetaStoreClient setting with
>> kerberized hive-site.xml.
>>
>> Any preconditions to met for the HiveMetaStoreClient to work correctly?
>>
>> On Thu, Jul 27, 2017 at 2:02 PM, wenxing zheng <wenxing.zh...@gmail.com>
>> wrote:
>>
>>> still didn't determine the root cause. And happened to find a JIRA
>>> related with my issue: https://issues.cloudera.org/browse/DISTRO-610.
>>>
>>>
>>>
>>> On Thu, Jul 27, 2017 at 11:41 AM, wenxing zheng <wenxing.zh...@gmail.com
>>> > wrote:
>>>
>>>> Thanks to Shkti. Will have a try immediately.
>>>>
>>>> On Thu, Jul 27, 2017 at 11:15 AM, shakti singh Shekhawat <
>>>> shaktisingh.shekhawa...@gmail.com> wrote:
>>>>
>>>>> Hi Wenxing,
>>>>>
>>>>> We recently had the same GSS Tgt issue when we moved to a Kerberized
>>>>> cluster. The solution that worked for us was "Create a file to define Java
>>>>> krb5login and name it as jaas.conf or jaas.java". Jaas authentication
>>>>> makes
>>>>> Java applications independent of underlying authentication technology.
>>>>>
>>>>> Please refer the below link from Oracle (or search for "How to add
>>>>> jaas configuration" in Google to see the 1st link in case the below link
>>>>> does not work) for your application.
>>>>> http://docs.oracle.com/javase/7/docs/technotes/guides/securi
>>>>> ty/jgss/tutorials/LoginConfigFile.html
>>>>>
>>>>> Thanks,
>>>>> Shakti Singh Shekhawat
>>>>>
>>>>> On Wed, Jul 26, 2017 at 10:42 PM wenxing zheng <
>>>>> wenxing.zh...@gmail.com> wrote:
>>>>>
>>>>>> Dear all,
>>>>>>
>>>>>> We have a Hive in 2.1.1 and a web application running against the
>>>>>> Hive server. Before enabling the Kerberos, everything is OK. But after
>>>>>> enabling the Kerberos, it always failed to do the authentication.
>>>>>>
>>>>>> - web application runs with: Jetty, hive client version: 1.2.1
>>>>>> and JDK 1.7
>>>>>> - Hive runs with JDK 1.8
>>>>>> - but both JDKs are running with JCE jars.
>>>>>>
>>>>>>
>>>>>> Followings are the errors:
>>>>>>
>>>>>>>
>>>>>>> 2017-07-27 10:29:16,622 INFO hive.metastore:Trying to connect to
>>>>>>> metastore with URI thrift://hdp-cli-01.dataservice.net:9083
>>>>>>> 2017-07-27 10:29:16,793 WARN
>>>>>>> org.apache.hadoop.util.NativeCodeLoader:Unable
>>>>>>> to load native-hadoop library for your platform... using builtin-java
>>>>>>> classes where applicable
>>>>>>> 2017-07-27 10:29:16,873 ERROR
>>>>>>> org.apache.thrift.transport.TSaslTransport:SASL
>>>>>>> negotiation failure
>>>>>>> javax.security.sasl.SaslException: GSS initiate failed [Caused by
>>>>>>> GSSException: No valid credentials provided (Mechanism level: Failed to
>>>>>>> find any Kerberos tgt)]
>>>>>>> at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChalleng
>>>>>>> e(GssKrb5Client.java:212)
>>>>>>> at org.apache.thrift.transport.TSaslClientTransport.handleSaslS
>>>>>>> tartMessage(TSaslClientTransport.java:94)
>>>>>>> at org.apache.thrift.transport.TSaslTransport.open(TSaslTranspo
>>>>>>> rt.java:271)
>>>>>>> at org.apache.thrift.transport.TSaslClientTransport.open(TSaslC
>>>>>>> lientTransport.java:37)
>>>>>>> at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1
>>>>>>> .run(TUGIAssumingTransport.java:52)
>>>>>>> at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1
>>>>>>> .run(TUGIAssumingTransport.java:49)
>>>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>>>> at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGro
>>>>>>> upInformation.java:1657)
>>>>>>> at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.o
>>>>>>> pen(TUGIAssumingTransport.java:49)
>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(Hi
>>>>>>> veMetaStoreClient.java:420)
>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(
>>>>>>> HiveMetaStoreClient.java:236)
>>>>>>> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(
>>>>>>> HiveMetaStoreClient.java:181)
>>>>>>> at com.taobao.zeus.store.CliTableManager.initClient(CliTableMan
>>>>>>> ager.java:60)
>>>>>>> at com.taobao.zeus.store.CliTableManager.<init>(CliTableManager
>>>>>>> .java:47)
>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>>>>>> Method)
>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>>>>>>> ConstructorAccessorImpl.java:57)
>>>>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>>>>>>> legatingConstructorAccessorImpl.java:45)
>>>>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>>>>>> at org.springframework.beans.BeanUtils.instantiateClass(BeanUti
>>>>>>> ls.java:100)
>>>>>>> at org.springframework.beans.factory.support.SimpleInstantiatio
>>>>>>> nStrategy.instantiate(SimpleInstantiationStrategy.java:61)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory.instantiateBean(AbstractAutowireCapableBean
>>>>>>> Factory.java:877)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory.createBeanInstance(AbstractAutowireCapableB
>>>>>>> eanFactory.java:839)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFac
>>>>>>> tory.java:440)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
>>>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory.createBean(AbstractAutowireCapableBeanFacto
>>>>>>> ry.java:380)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y$1.getObject(AbstractBeanFactory.java:264)
>>>>>>> at org.springframework.beans.factory.support.DefaultSingletonBe
>>>>>>> anRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y.doGetBean(AbstractBeanFactory.java:261)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y.getBean(AbstractBeanFactory.java:185)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y.getBean(AbstractBeanFactory.java:164)
>>>>>>> at org.springframework.beans.factory.support.DefaultListableBea
>>>>>>> nFactory.findAutowireCandidates(DefaultListableBeanFactory.java:671)
>>>>>>> at org.springframework.beans.factory.support.DefaultListableBea
>>>>>>> nFactory.resolveDependency(DefaultListableBeanFactory.java:610)
>>>>>>> at org.springframework.beans.factory.annotation.AutowiredAnnota
>>>>>>> tionBeanPostProcessor$AutowiredFieldElement.inject(Autowired
>>>>>>> AnnotationBeanPostProcessor.java:412)
>>>>>>> at org.springframework.beans.factory.annotation.InjectionMetada
>>>>>>> ta.injectFields(InjectionMetadata.java:105)
>>>>>>> at org.springframework.beans.factory.annotation.AutowiredAnnota
>>>>>>> tionBeanPostProcessor.postProcessAfterInstantiation(Autowire
>>>>>>> dAnnotationBeanPostProcessor.java:240)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory.populateBean(AbstractAutowireCapableBeanFac
>>>>>>> tory.java:959)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFac
>>>>>>> tory.java:472)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
>>>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>>>> at org.springframework.beans.factory.support.AbstractAutowireCa
>>>>>>> pableBeanFactory.createBean(AbstractAutowireCapableBeanFacto
>>>>>>> ry.java:380)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y$1.getObject(AbstractBeanFactory.java:264)
>>>>>>> at org.springframework.beans.factory.support.DefaultSingletonBe
>>>>>>> anRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y.doGetBean(AbstractBeanFactory.java:261)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y.getBean(AbstractBeanFactory.java:185)
>>>>>>> at org.springframework.beans.factory.support.AbstractBeanFactor
>>>>>>> y.getBean(AbstractBeanFactory.java:164)
>>>>>>> at org.springframework.beans.factory.support.DefaultListableBea
>>>>>>> nFactory.preInstantiateSingletons(DefaultListableBeanFactory
>>>>>>> .java:429)
>>>>>>> at org.springframework.context.support.AbstractApplicationConte
>>>>>>> xt.finishBeanFactoryInitialization(AbstractApplicationContex
>>>>>>> t.java:728)
>>>>>>> at org.springframework.context.support.AbstractApplicationConte
>>>>>>> xt.refresh(AbstractApplicationContext.java:380)
>>>>>>> at org.springframework.web.context.ContextLoader.createWebAppli
>>>>>>> cationContext(ContextLoader.java:255)
>>>>>>> at org.springframework.web.context.ContextLoader.initWebApplica
>>>>>>> tionContext(ContextLoader.java:199)
>>>>>>> at org.springframework.web.context.ContextLoaderListener.contex
>>>>>>> tInitialized(ContextLoaderListener.java:45)
>>>>>>> at org.eclipse.jetty.server.handler.ContextHandler.callContextI
>>>>>>> nitialized(ContextHandler.java:800)
>>>>>>> at org.eclipse.jetty.servlet.ServletContextHandler.callContextI
>>>>>>> nitialized(ServletContextHandler.java:444)
>>>>>>> at org.eclipse.jetty.server.handler.ContextHandler.startContext
>>>>>>> (ContextHandler.java:791)
>>>>>>> at org.eclipse.jetty.servlet.ServletContextHandler.startContext
>>>>>>> (ServletContextHandler.java:294)
>>>>>>> at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppCon
>>>>>>> text.java:1349)
>>>>>>> at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppCo
>>>>>>> ntext.java:1342)
>>>>>>> at org.eclipse.jetty.server.handler.ContextHandler.doStart(Cont
>>>>>>> extHandler.java:741)
>>>>>>> at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext
>>>>>>> .java:505)
>>>>>>> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(Abs
>>>>>>> tractLifeCycle.java:68)
>>>>>>> at org.eclipse.jetty.deploy.bindings.StandardStarter.processBin
>>>>>>> ding(StandardStarter.java:41)
>>>>>>> at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCyc
>>>>>>> le.java:186)
>>>>>>> at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(De
>>>>>>> ploymentManager.java:498)
>>>>>>> at org.eclipse.jetty.deploy.DeploymentManager.addApp(Deployment
>>>>>>> Manager.java:146)
>>>>>>> at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileA
>>>>>>> dded(ScanningAppProvider.java:180)
>>>>>>> at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(
>>>>>>> WebAppProvider.java:440)
>>>>>>> at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fil
>>>>>>> eAdded(ScanningAppProvider.java:64)
>>>>>>> at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:609)
>>>>>>> at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.jav
>>>>>>> a:528)
>>>>>>> at org.eclipse.jetty.util.Scanner.scan(Scanner.java:391)
>>>>>>> at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:313)
>>>>>>> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(Abs
>>>>>>> tractLifeCycle.java:68)
>>>>>>> at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doSta
>>>>>>> rt(ScanningAppProvider.java:150)
>>>>>>> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(Abs
>>>>>>> tractLifeCycle.java:68)
>>>>>>> at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(
>>>>>>> DeploymentManager.java:560)
>>>>>>> at org.eclipse.jetty.deploy.DeploymentManager.doStart(Deploymen
>>>>>>> tManager.java:235)
>>>>>>> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(Abs
>>>>>>> tractLifeCycle.java:68)
>>>>>>> at org.eclipse.jetty.util.component.ContainerLifeCycle.start(Co
>>>>>>> ntainerLifeCycle.java:132)
>>>>>>> at org.eclipse.jetty.server.Server.start(Server.java:387)
>>>>>>> at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(
>>>>>>> ContainerLifeCycle.java:114)
>>>>>>> at org.eclipse.jetty.server.handler.AbstractHandler.doStart(Abs
>>>>>>> tractHandler.java:61)
>>>>>>> at org.eclipse.jetty.server.Server.doStart(Server.java:354)
>>>>>>> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(Abs
>>>>>>> tractLifeCycle.java:68)
>>>>>>> at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguratio
>>>>>>> n.java:1255)
>>>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>>>> at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration
>>>>>>> .java:1174)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>>>>>> ssorImpl.java:57)
>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>>>>>> thodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>> at org.eclipse.jetty.start.Main.invokeMain(Main.java:321)
>>>>>>> at org.eclipse.jetty.start.Main.start(Main.java:817)
>>>>>>> at org.eclipse.jetty.start.Main.main(Main.java:112)
>>>>>>> Caused by: GSSException: No valid credentials provided (Mechanism
>>>>>>> level: Failed to find any Kerberos tgt)
>>>>>>> at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5In
>>>>>>> itCredential.java:147)
>>>>>>> at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(
>>>>>>> Krb5MechFactory.java:121)
>>>>>>> at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(K
>>>>>>> rb5MechFactory.java:187)
>>>>>>> at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSMana
>>>>>>> gerImpl.java:223)
>>>>>>> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextIm
>>>>>>> pl.java:212)
>>>>>>> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextIm
>>>>>>> pl.java:179)
>>>>>>> at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChalleng
>>>>>>> e(GssKrb5Client.java:193)
>>>>>>> ... 94 more
>>>>>>
>>>>>>
>>>>>> Appreciated for your advice.
>>>>>> Kind Regards, Wenxing
>>>>>>
>>>>>
>>>>
>>>
>>
>
