Yongle Zhang created HIVE-24493:
-----------------------------------
Summary: after upgrading Hive from 2.1.1 to 2.3.7, log keeps
printing "Hive Schema version 2.3.0 does not match metastore's schema version
2.1.0"
Key: HIVE-24493
URL: https://issues.apache.org/jira/browse/HIVE-24493
Project: Hive
Issue Type: Bug
Environment: {code:java}
Linux version 4.15.0-112-generic (buildd@lcy01-amd64-027) (gcc version 7.5.0
(Ubuntu 7.5.0-3ubuntu1~18.04)) #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020
{code}
{code:java}
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode){code}
Reporter: Yongle Zhang
We tried to upgrade a hive single node from 2.1.1 to 2.3.7 and use derby as
the database. After upgrade, Hive log keeps printing the following Error
message repeatedly:
{code:java}
// code placeholder
2020-12-05T02:47:26,573 ERROR [main] metastore.RetryingHMSHandler: HMSHandler
Fatal error: MetaException(message:Hive Schema version 2.3.0 does not match
metastore's schema version 2.1.0 Metastore is not upgraded or corrupt)
at
org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7825)
at
org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7788)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
at com.sun.proxy.$Proxy34.verifySchema(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:595)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:588)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:655)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:431)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:79)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6902)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:164)
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1707)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3600)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3652)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3632)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3894)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:248)
at
org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:388)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:332)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288)
at
org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:917)
at
org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:881)
at
org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1687)
at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:130)
at org.apache.hive.service.cli.CLIService.init(CLIService.java:114)
at org.apache.hive.service.CompositeService.init(CompositeService.java:59)
at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:142)
at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:607)
at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:100)
at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:855)
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:724)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153){code}
We did use schematool to upgrade Hive. The log also shows that upgrade
metastore is performed. Here are the steps to reproduce it:
1. Start up a standalone hive node of version 2.1.1 using embedded derby, and
use schematool to initialize the schema.
2. Use the following shell script to generate 20k data.
{code:java}
rm -rf ./data.txt
touch data.txt
for((i=0;i<20000;i++))
do
str=',name';
name=${i}${str}${i}
echo $name>> data.txt
done
{code}
3. Create a table and load data by using:
{code:java}
create table testTable(id int,name string) row format delimited fields
terminated by ',';
LOAD DATA LOCAL INPATH '/data.txt' OVERWRITE INTO TABLE testTable;
{code}
4. We stopped the nodes, and started a node running hive compiled from 2.3.7,
while keeping the node of hadoop running.
5. Run the following command to upgrade schema.
{code:java}
schematool -dbType derby -upgradeSchema
{code}
Log shows that metastore is being upgraded:
{code:java}
LF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:
jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting upgrade metastore schema from version 2.1.0 to 2.3.0
Upgrade script upgrade-2.1.0-to-2.2.0.derby.sql
Connecting to jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
Connected to: Apache Derby (version 10.10.2.0 - (1582446))
Driver: Apache Derby Embedded JDBC Driver (version 10.10.2.0 - (1582446))
.
.
.
beeline> Completed upgrade-2.1.0-to-2.2.0.derby.sql
Upgrade script upgrade-2.2.0-to-2.3.0.derby.sql
Connecting to jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
Connected to: Apache Derby (version 10.10.2.0 - (1582446))
Driver: Apache Derby Embedded JDBC Driver (version 10.10.2.0 - (1582446))
.
.
.
beeline> Completed upgrade-2.2.0-to-2.3.0.derby.sql
schemaTool completed
{code}
6. However, after upgrade, the log still keeps printing the previous Error
message with exception - "Hive Schema version 2.3.0 does not match metastore's
schema version 2.1.0”.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
