Dear Doris Community,
I am reaching out to the community to seek assistance with an integration
issue I've encountered while trying to use Hive on a CDP distribution with
Doris 2.1.3
Here are the details of my setup:
Doris Version: 2.1.3
Hive Version: 3.1.3
Cloudera Version: 7.4.4
I have successfully created a Hive catalog in Doris and have been able to
set up the integration between Hive and Doris. However, when I attempt to
query a Hive internal table using Doris, I encounter the following error:
[image: image.png]
ERROR 1105 (HY000): errCode = 2, detailMessage = get file split failed for
table: baby1, err: java.lang.Exception: Hive 2.x versioned full-acid tables
need to run major compaction.
I would greatly appreciate any insights, guidance, or solutions that the
Doris community can offer to help me resolve this issue. If anyone has
encountered a similar problem or has expertise in integrating cloudera Hive
with Doris, your assistance would be invaluable.
Thank you in advance for your time and support. I look forward to hearing
from the community and working together to find a solution to this
challenge.
[image: image.png]
2024-06-03 12:10:33,195 INFO
org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAccessController:
[pool-5-thread-25]: Created SQLStdHiveAccessController for session context :
HiveAuthzSessionContext [sessionString=HiveMetaStore, clientType=HIVEMETASTORE]
2024-06-03 12:10:33,507 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-25]: Skipping translation for processor with null
2024-06-03 12:10:33,679 INFO org.apache.hadoop.hive.metastore.ObjectStore:
[pool-5-thread-29]: RawStore:
org.apache.hadoop.hive.metastore.ObjectStore@697f5d5c, with PersistenceManager:
null will be shutdown
2024-06-03 12:10:33,679 INFO org.apache.hadoop.hive.metastore.ObjectStore:
[pool-5-thread-29]: RawStore:
org.apache.hadoop.hive.metastore.ObjectStore@697f5d5c, with PersistenceManager:
org.datanucleus.api.jdo.JDOPersistenceManager@1fbedb8c created in the thread
with id: 10105
2024-06-03 12:10:33,683 ERROR
org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-5-thread-24]:
java.lang.NumberFormatException: For input string: "9-21"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2024-06-03 12:10:33,685 ERROR org.apache.thrift.server.TThreadPoolServer:
[pool-5-thread-24]: Error occurred during processing of message.
java.lang.NumberFormatException: For input string: "9-21"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_232]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_232]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_232]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
2024-06-03 12:10:33,689 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-24]: 37: Cleaning up thread local RawStore...
2024-06-03 12:10:33,690 INFO
org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-24]:
ugi=hive ip=10.0.0.35cmd=Cleaning up thread local RawStore...
2024-06-03 12:10:33,690 INFO org.apache.hadoop.hive.metastore.ObjectStore:
[pool-5-thread-24]: RawStore:
org.apache.hadoop.hive.metastore.ObjectStore@23a9e56c, with PersistenceManager:
org.datanucleus.api.jdo.JDOPersistenceManager@57ebb3d0 will be shutdown
2024-06-03 12:10:33,690 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-24]: 37: Done cleaning up thread local RawStore
2024-06-03 12:10:33,690 INFO
org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-24]:
ugi=hive ip=10.0.0.35cmd=Done cleaning up thread local RawStore
2024-06-03 12:10:33,708 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-29]: Created RawStore:
org.apache.hadoop.hive.metastore.ObjectStore@697f5d5c from thread id: 10105
2024-06-03 12:10:34,699 ERROR
org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-5-thread-30]:
java.lang.NumberFormatException: For input string: "9-21"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2024-06-03 12:10:34,701 ERROR org.apache.thrift.server.TThreadPoolServer:
[pool-5-thread-30]: Error occurred during processing of message.
java.lang.NumberFormatException: For input string: "9-21"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_232]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_232]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_232]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
2024-06-03 12:10:34,704 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-30]: 43: Done cleaning up thread local RawStore
2024-06-03 12:10:34,704 INFO
org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-30]:
ugi=hive ip=10.0.0.35cmd=Done cleaning up thread local RawStore
2024-06-03 12:10:34,849 INFO org.apache.hadoop.hive.metastore.txn.TxnHandler:
[pool-5-thread-22]: Added entries to MIN_HISTORY_LEVEL with a single query for
current txn: [22]
2024-06-03 12:10:34,950 ERROR
org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-5-thread-26]:
java.lang.NumberFormatException: For input string: "9-22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2024-06-03 12:10:34,951 ERROR org.apache.thrift.server.TThreadPoolServer:
[pool-5-thread-26]: Error occurred during processing of message.
java.lang.NumberFormatException: For input string: "9-22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_232]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_232]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_232]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
2024-06-03 12:10:34,955 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-26]: 39: Cleaning up thread local RawStore...
2024-06-03 12:10:34,955 INFO
org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-26]:
ugi=hive ip=10.0.0.35cmd=Cleaning up thread local RawStore...
2024-06-03 12:10:34,955 INFO org.apache.hadoop.hive.metastore.ObjectStore:
[pool-5-thread-26]: RawStore:
org.apache.hadoop.hive.metastore.ObjectStore@eeab62a, with PersistenceManager:
org.datanucleus.api.jdo.JDOPersistenceManager@1ebcab3e will be shutdown
2024-06-03 12:10:34,955 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-26]: 39: Done cleaning up thread local RawStore
2024-06-03 12:10:34,956 INFO
org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-26]:
ugi=hive ip=10.0.0.35cmd=Done cleaning up thread local RawStore
2024-06-03 12:10:35,963 ERROR
org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-5-thread-31]:
java.lang.NumberFormatException: For input string: "9-22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2024-06-03 12:10:35,965 ERROR org.apache.thrift.server.TThreadPoolServer:
[pool-5-thread-31]: Error occurred during processing of message.
java.lang.NumberFormatException: For input string: "9-22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_232]
at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.common.ValidReadTxnList.readFromString(ValidReadTxnList.java:167)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.common.ValidReadTxnList.<init>(ValidReadTxnList.java:56)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getValidWriteIds(TxnHandler.java:1435)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_valid_write_ids(HiveMetaStore.java:8423)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at com.sun.proxy.$Proxy28.get_valid_write_ids(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20160)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_valid_write_ids.getResult(ThriftHiveMetastore.java:20144)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_232]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_232]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_232]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
2024-06-03 12:10:35,968 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
[pool-5-thread-31]: 44: Done cleaning up thread local RawStore
2024-06-03 12:10:35,968 INFO
org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-31]:
ugi=hive ip=10.0.0.35cmd=Done cleaning up thread local RawStore
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
