Here after a link related to hive3: http://www.adaltas.com/fr/2019/07/25/hive-3-fonctionnalites-conseils-astuces/ The author suggests the use of the hive user. So the hive keytab to execute merge query due to kerberos on my side. Is it the only way to make compaction work properly ?
Le lun. 26 août 2019 à 07:51, David Morin <morin.david....@gmail.com> a écrit : > Hello, > I've been trying "ALTER TABLE (table_name) COMPACT 'MAJOR'" on my Hive 2 > environment, but it always fails (HDP 2.6.5 precisely). It seems that the > merged base file is created but the delta is not deleted. > I found that it was because the HiveMetastore Client can't connect to the > metastore because of GSS Initiate failed during cleaning step. > > Here after the logs on the Metastore side (DEBUG mode activated): > > 2019-08-25 21:42:51,729 INFO [Thread-24]: hive.metastore > *(HiveMetaStoreClient.java:open(451)) > - Trying to connect to metastore with URI thrift://XXX:9083* > 2019-08-25 21:42:51,747 DEBUG [Thread-24]: security.UserGroupInformation > *(UserGroupInformation.java:logPrivilegedAction(1896)) > - PrivilegedAction as:myuser (auth:PROXY) via hive/XXXX (auth:KERBEROS) * > from:org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > 2019-08-25 21:42:51,748 DEBUG [Thread-24]: transport.TSaslTransport > (TSaslTransport.java:open(261)) - opening transport > org.apache.thrift.transport.TSaslClientTransport@58fdf6d4 > 2019-08-25 21:42:51,750 ERROR [Thread-24]: transport.TSaslTransport > (TSaslTransport.java:open(315)) - SASL negotiation failure > *javax.security.sasl.SaslException: GSS initiate failed [Caused by > GSSException: No valid credentials provided (Mechanism level: Failed to > find any Kerberos tgt)]* > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) > at > org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:495) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:290) > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3641) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3675) > at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3917) > at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:225) > at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:217) > at > org.apache.hadoop.hive.ql.txn.compactor.Cleaner.removeFiles(Cleaner.java:344) > at > org.apache.hadoop.hive.ql.txn.compactor.Cleaner.access$000(Cleaner.java:63) > at org.apache.hadoop.hive.ql.txn.compactor.Cleaner$1.run(Cleaner.java:295) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) > at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.clean(Cleaner.java:292) > at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:162) > Caused by: GSSException: No valid credentials provided (Mechanism level: > Failed to find any Kerberos tgt) > at > sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) > at > sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) > at > sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) > at > sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) > at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) > at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) > ... 33 more > 2019-08-25 21:42:51,763 DEBUG [Thread-24]: transport.TSaslTransport > (TSaslTransport.java:sendSaslMessage(162)) - CLIENT: Writing message with > status BAD and payload length 19 > 2019-08-25 21:42:51,764 WARN [Thread-24]: hive.metastore > *(HiveMetaStoreClient.java:open(505)) - Failed to connect to the MetaStore > Server...org.apache.thrift.transport.TTransportException: GSS initiate > failed* > at > org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) > at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:495) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:290) > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3641) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3675) > at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3917) > at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:225) > at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:217) > > > * at > org.apache.hadoop.hive.ql.txn.compactor.Cleaner.removeFiles(Cleaner.java:344) > at > org.apache.hadoop.hive.ql.txn.compactor.Cleaner.access$000(Cleaner.java:63) > at org.apache.hadoop.hive.ql.txn.compactor.Cleaner$1.run(Cleaner.java:295)* > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) > at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.clean(Cleaner.java:292) > at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:162) > 2019-08-25 21:42:51,764 INFO [Thread-24]: hive.metastore > (HiveMetaStoreClient.java:open(539)) - Waiting 5 seconds before next > connection attempt. > 2019-08-25 21:42:51,766 DEBUG [pool-7-thread-1]: > security.UserGroupInformation > *(UserGroupInformation.java:logPrivilegedAction(1896)) > - PrivilegedAction as:hive/XXXXX (auth:KERBEROS) * > from:org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:606) > 2019-08-25 21:42:51,769 DEBUG [pool-7-thread-1]: > transport.TSaslServerTransport > (TSaslServerTransport.java:getTransport(213)) - transport map does not > contain key > 2019-08-25 21:42:51,769 DEBUG [pool-7-thread-1]: transport.TSaslTransport > (TSaslTransport.java:open(261)) - opening transport > org.apache.thrift.transport.TSaslServerTransport@327e3ff0 > 2019-08-25 21:42:51,771 DEBUG [pool-7-thread-1]: > transport.TSaslServerTransport > (TSaslServerTransport.java:getTransport(218)) - failed to open server > transport > org.apache.thrift.transport.TTransportException: Peer indicated failure: > GSS initiate failed > at > org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199) > at > org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125) > at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:609) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:606) > at java.security.AccessController.doPrivileged(Native Method) >
