[
https://issues.apache.org/jira/browse/HIVE-23286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marta Kuczora updated HIVE-23286:
---------------------------------
Description:
In FileSinkOperator there is a code path when the operator is aborted:
{noformat}
} else {
// Will come here if an Exception was thrown in map() or reduce().
// Hadoop always call close() even if an Exception was thrown in map() or
// reduce().
for (FSPaths fsp : valToPaths.values()) {
fsp.abortWritersAndUpdaters(fs, abort, !autoDelete && isNativeTable()
&& !conf.isMmTable());
}
{noformat}
In this part, the fsp.abortWritersAndUpdaters method call should consider the
conf.isDirectInsert parameter as well. Since this parameter is missing, this
method can delete the content of the table if an insert failure abort the
FileSinkOperator and the ACID direct insert it turned on.
was:
In FileSinkOperator there is a code path when the operator is aborted:
{noformat}
} else {
// Will come here if an Exception was thrown in map() or reduce().
// Hadoop always call close() even if an Exception was thrown in map() or
// reduce().
for (FSPaths fsp : valToPaths.values()) {
fsp.abortWritersAndUpdaters(fs, abort, !autoDelete && isNativeTable()
&& !conf.isMmTable());
}
{noformat}
In this part, the fsp.abortWritersAndUpdaters method call should consider the
conf.isDirectInsert parameter as well. Since this parameter is missing, this
method can delete the content of the table.
> The clean-up in case of an aborted FileSinkOperator is not correct for ACID
> direct insert
> -----------------------------------------------------------------------------------------
>
> Key: HIVE-23286
> URL: https://issues.apache.org/jira/browse/HIVE-23286
> Project: Hive
> Issue Type: Bug
> Reporter: Marta Kuczora
> Assignee: Marta Kuczora
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23286.1.patch
>
>
> In FileSinkOperator there is a code path when the operator is aborted:
> {noformat}
> } else {
> // Will come here if an Exception was thrown in map() or reduce().
> // Hadoop always call close() even if an Exception was thrown in map()
> or
> // reduce().
> for (FSPaths fsp : valToPaths.values()) {
> fsp.abortWritersAndUpdaters(fs, abort, !autoDelete && isNativeTable()
> && !conf.isMmTable());
> }
> {noformat}
> In this part, the fsp.abortWritersAndUpdaters method call should consider the
> conf.isDirectInsert parameter as well. Since this parameter is missing, this
> method can delete the content of the table if an insert failure abort the
> FileSinkOperator and the ACID direct insert it turned on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
