[
https://issues.apache.org/jira/browse/IMPALA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17905939#comment-17905939
]
Quanlong Huang commented on IMPALA-13065:
-----------------------------------------
With the service commands added by IMPALA-13064, the steps might be much easier.
It'd be nice to note some configuration changes in the wiki. E.g. I recently
encounted an issue of launching catalogd.
{noformat}
E1216 00:23:21.448288 1250 JniCatalog.java:229] Unable to fetch the current
notification event id from metastore.
Java exception follows:
org.apache.thrift.TApplicationException: Internal error processing
get_current_notificationEventId
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_current_notificationEventId(ThriftHiveMetastore.java:7100)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_current_notificationEventId(ThriftHiveMetastore.java:7088)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getCurrentNotificationEventId(HiveMetaStoreClient.java:4395)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
at com.sun.proxy.$Proxy11.getCurrentNotificationEventId(Unknown Source)
at
org.apache.impala.service.JniCatalog.getEventsProcessor(JniCatalog.java:225)
at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:179)
I1216 00:23:21.448652 1250 jni-util.cc:321]
org.apache.impala.catalog.CatalogException: Fatal error while initializing
metastore event processor
at
org.apache.impala.service.JniCatalog.getEventsProcessor(JniCatalog.java:230)
at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:179)
Caused by: org.apache.thrift.TApplicationException: Internal error processing
get_current_notificationEventId
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_current_notificationEventId(ThriftHiveMetastore.java:7100)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_current_notificationEventId(ThriftHiveMetastore.java:7088)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getCurrentNotificationEventId(HiveMetaStoreClient.java:4395)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
at com.sun.proxy.$Proxy11.getCurrentNotificationEventId(Unknown Source)
at
org.apache.impala.service.JniCatalog.getEventsProcessor(JniCatalog.java:225)
... 1 more {noformat}
Checking the HMS logs, it's due to a permission issue
{noformat}
2024-12-16T00:23:20,424 ERROR [TThreadPoolServer WorkerProcess-12]
metastore.HiveMetaStore: Not authorized to make the
get_current_notificationEventId call. You can try to disable
metastore.metastore.event.db.notification.api.auth
org.apache.hadoop.hive.metastore.api.MetaException: User impala is not allowed
to perform this API call
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.authorizeProxyPrivilege(HiveMetaStore.java:9919)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_current_notificationEventId(HiveMetaStore.java:9879)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_412]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_412]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:160)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:121)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at com.sun.proxy.$Proxy34.get_current_notificationEventId(Unknown
Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult(ThriftHiveMetastore.java:24934)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult(ThriftHiveMetastore.java:24914)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
~[libthrift-0.16.0.jar:0.16.0]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_412]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_412]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1910)
~[hadoop-common-3.1.1.7.3.1.0-160.jar:?]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
~[hive-standalone-metastore-3.1.3000.7.3.1.0-160.jar:3.1.3000.7.3.1.0-160]
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
~[libthrift-0.16.0.jar:0.16.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_412]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_412]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_412]{noformat}
The workaround is setting metastore.metastore.event.db.notification.api.auth to
false in hive-site.xml of HMS. Or add the "impala" user to superusers defined
in the Hadoop proxy user settings.
https://github.com/apache/hive/blob/4898d7bfaa1437d39d25070e0ab63c5e0e0de9d7/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java#L837-L838
> Introduce package scripts to launch Impala processes
> ----------------------------------------------------
>
> Key: IMPALA-13065
> URL: https://issues.apache.org/jira/browse/IMPALA-13065
> Project: IMPALA
> Issue Type: Documentation
> Reporter: Quanlong Huang
> Assignee: XiangYang
> Priority: Major
>
> We should add document for how to use the scripts installed by the RPM/DEB
> packages at
> https://impala.apache.org/docs/build/html/topics/impala_processes.html
> CC [~yx91490]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
