Yongle Zhang created HIVE-24440:
-----------------------------------
Summary: Upgrading Hive from 2.3.x(2.3.7) to 3.x(3.0.0), schema
failed to upgrade
Key: HIVE-24440
URL: https://issues.apache.org/jira/browse/HIVE-24440
Project: Hive
Issue Type: Bug
Affects Versions: 3.0.0, 2.3.7
Reporter: Yongle Zhang
We tried to upgrade a hive single node from 2.3.7 to 3.0.0 and schema failed
to upgrade because of a truncation error encountered when trying to shrink CHAR
'2020-11-26 00:04:29.215' to length 1:
Error message:
{code:java}
0: jdbc:derby:> UPDATE "APP"."TBLS" SET "IS_REWRITE_ENABLED" = CURRENT_TIMESTAMP
Error: A truncation error was encountered trying to shrink CHAR '2020-11-26
00:04:29.215' to length 1. (state=22001,code=30000)
Closing: 0: jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore
state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore
state would be inconsistent !!
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:548)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1443)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.io.IOException: Schema script failed, errorcode 2
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1165)
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1143)
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:543)
... 7 more
*** schemaTool failed ***
{code}
It can be reproduced through the following steps:
# Start up a standalone hive node of version 2.3.7 using embedded derby, and
use schematool to initialize the schema.
# Use the following shell script to generate 20k data.
{code:java}
rm -rf ./data.txt
touch data.txt
for((i=0;i<20000;i++))
do
str=',name';
name=${i}${str}${i}
echo $name>> data.txt
done{code}
# Create a table and load data by using:
{code:java}
create table testTable(id int,name string) row format delimited fields
terminated by ',';
LOAD DATA LOCAL INPATH '/data.txt' OVERWRITE INTO TABLE testTable;
{code}
# stop hive 2.3.7, and start hive compiled from 3.0.0, while keeping hadoop
running.
# Run the following command to upgrade schema. Then we will see the previous
error message.
{code:java}
schematool -dbType derby -upgradeSchema --verbose
{code}
# If we use the schematool to upgrade schema again, we will get the following
error message:
{code:java}
0: jdbc:derby:> CREATE TABLE "APP"."METASTORE_DB_PROPERTIES" ("PROPERTY_KEY"
VARCHAR(255) NOT NULL, "PROPERTY_VALUE" VARCHAR(1000) NOT NULL, "DESCRIPTION"
VARCHAR(1000))
Error: Table/View 'METASTORE_DB_PROPERTIES' already exists in Schema 'APP'.
(state=X0Y32,code=30000)
Closing: 0: jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore
state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore
state would be inconsistent !!
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:548)
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:512)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1440)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.io.IOException: Schema script failed, errorcode 2
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1165)
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1143)
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:543)
... 8 more
*** schemaTool failed ***
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
