[
https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985666#action_12985666
]
Chinna Rao Lalam commented on HIVE-1897:
----------------------------------------
In HiveAlterHandler ( alterTable ) , if the HDFS operation failed then its
better to rename the table to the old table name. To avoid the inconsistency
b/w Metadata and HDFS data.
h4. Old code snippet
h4.
{code}
if (success && moveData) {
// change the file name in hdfs
// check that src exists otherwise there is no need to copy the data
try {
if (srcFs.exists(srcPath)) {
// rename the src to destination
srcFs.rename(srcPath, destPath);
}
} catch (IOException e) {
throw new InvalidOperationException("Unable to access old
location "
+ srcPath + " for table " + dbname + "." + name);
}
{code}
h4. Changes Proposed
Before throwing the "InvalidOperationException" , rename the table name to the
old table name.
{code}
if (success && moveData) {
// change the file name in hdfs
// check that src exists otherwise there is no need to copy the data
try {
if (srcFs.exists(srcPath)) {
// rename the src to destination
srcFs.rename(srcPath, destPath);
}
} catch (IOException e) {
boolean revertMetaDataTransaction = false;
try
{
msdb.openTransaction();
// Alter the table name to have the old table name.
msdb.alterTable(dbname, newt.getTableName(), oldt);
revertMetaDataTransaction = msdb.commitTransaction();
}
catch (Exception e1)
{
LOG.error("Reverting metadata opeation failed During HDFS operation
failed",e1);
if(!revertMetaDataTransaction)
{
msdb.rollbackTransaction();
}
}
throw new InvalidOperationException("Unable to access old location "
+ srcPath + " for table " + dbname + "." + name);
}
}
{code}
> Alter command execution "when HDFS is down" results in holding stale data in
> MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise
> Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to
> secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location
> hdfs://localhost:9000/user/hive/warehouse/firsttable for table
> default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client
> gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed
> successfully and then *"DFS"* operation is started. In this scenario, *"DFS"*
> is down. As a result, execution of the query failed and partial information
> of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
