[
https://issues.apache.org/jira/browse/HIVE-26211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ECFuzz updated HIVE-26211:
--------------------------
Priority: Blocker (was: Major)
> "hive.server2.webui.max.historic.queries" should be avoided to be set too
> large, otherwise it will cause blocking
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-26211
> URL: https://issues.apache.org/jira/browse/HIVE-26211
> Project: Hive
> Issue Type: Bug
> Components: Clients
> Affects Versions: 3.1.2
> Environment: Hive 3.1.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_162
> java.vendor=Oracle Corporation
> Reporter: ECFuzz
> Assignee: ECFuzz
> Priority: Blocker
>
> In hive-default.xml.template
> {code:java}
> <property>
> <name>hive.server2.webui.max.historic.queries</name>
> <value>25</value>
> <description>The maximum number of past queries to show in HiverSever2
> WebUI.</description>
> </property>{code}
> Set hive.server2.webui.max.historic.queries to a relatively large value, take
> 20000000 as an example, start hiveserver2, it can start hiveserver normally,
> and logging without exception.
> {code:java}
> liky@ljq1:/usr/local/hive/conf$ hiveserver2
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> 2022-05-09 20:03:41: Starting HiveServer2
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
> Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d{code}
> Next, if you use beeline as a client to connect to hive and send a request
> for database related operations, for example, if you query all the databases,
> after successfully executing "show databases", beeline blocks and no other
> operations can be performed.
> {code:java}
> liky@ljq1:/opt/hive$ beeline
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Beeline version 3.1.2 by Apache Hive
> beeline> !connect jdbc:hive2://192.168.1.194:10000/default
> Connecting to jdbc:hive2://192.168.1.194:10000/default
> Enter username for jdbc:hive2://192.168.1.194:10000/default: hive
> Enter password for jdbc:hive2://192.168.1.194:10000/default: *****
> Connected to: Apache Hive (version 3.1.2)
> Driver: Hive JDBC (version 3.1.2)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> 0: jdbc:hive2://192.168.1.194:10000/default> show databases
> . . . . . . . . . . . . . . . . . . . . . .> ;
> INFO : Compiling
> command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b):
> show databases
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:database_name, type:string,
> comment:from deserializer)], properties:null)
> INFO : Completed compiling
> command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b);
> Time taken: 0.393 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Executing
> command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b):
> show databases
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b);
> Time taken: 0.109 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> ----------------
> database_name
> ----------------
> default
> ----------------
> 1 row selected (1.374 seconds)
> {code}
> Also, on the hiveserver side, a runtime null pointer exception is thrown, and
> the observation log throws no warnings or errors.
> {code:java}
> liky@ljq1:/usr/local/hive/conf$ hiveserver2
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Error: Could not find or load main class
> org.apache.hadoop.hbase.util.GetJavaProperty
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hbase-2.2.2/hbase/maven/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> 2022-05-09 20:03:41: Starting HiveServer2
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
> Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d
> OK
> Exception in thread "HiveServer2-Handler-Pool: Thread-41"
> java.lang.RuntimeException: java.lang.NullPointerException
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
> at com.sun.proxy.$Proxy42.close(Unknown Source)
> at
> org.apache.hive.service.cli.session.SessionManager.closeSession(SessionManager.java:552)
> at
> org.apache.hive.service.cli.CLIService.closeSession(CLIService.java:241)
> at
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService$1.deleteContext(ThriftBinaryCLIService.java:141)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hive.service.cli.operation.OperationManager.getQueryId(OperationManager.java:189)
> at
> org.apache.hive.service.cli.operation.OperationManager.removeOperation(OperationManager.java:206)
> at
> org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationManager.java:287)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.close(HiveSessionImpl.java:751)
> at
> org.apache.hive.service.cli.session.HiveSessionImplwithUGI.close(HiveSessionImplwithUGI.java:93)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> ... 14 more{code}
> If you open another beeline as a client to initiate a database operation
> request to the hiveserver, the beeline will also block after executing a
> request and the hiveserver will throw the same exception again.
> If you call java API write a program to establish a jdbc connection as a
> client to access hive, take JDBCDemo as an example.
> {code:java}
> import demo.utils.JDBCUtils;
> public class JDBCDemo{
> public static void main(String[] args) throws Exception {
> JDBCUtils.init();
> JDBCUtils.createDatabase();
> JDBCUtils.showDatabases();
> JDBCUtils.createTable();
> JDBCUtils.showTables();
> JDBCUtils.descTable();
> JDBCUtils.loadData();
> JDBCUtils.selectData();
> JDBCUtils.countData();
> JDBCUtils.dropDatabase();
> JDBCUtils.dropTable();
> JDBCUtils.destory();
> }
> }
> {code}
> Starting the hiveserver and running the written program will block directly
> after the first database operation request, and the terminal will not throw
> any hints, in line with beeline's blocking node, the hiveserver throws the
> same exceptions as before.
> {code:java}
> liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Running: drop database if exists hive_jdbc_test
> Running: create database hive_jdbc_test
>
>
>
> ^Cliky@ljq1:~/hive_jdbc_test$
> {code}
> In summary, when the value of hive.server2.webui.max.historic.queries is set
> to a larger value, {*}it blocks all clients that initiate the request{*}, and
> the hive server side only throws a runtime null pointer exception, and the
> observation log throws no related warnings and errors, causing some impact.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
