[
https://issues.apache.org/jira/browse/HIVE-27899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenyu Zheng updated HIVE-27899:
--------------------------------
Description: As I mentioned in HIVE-25561, when tez turns on speculative
execution, the data file produced by hive may be duplicated. I mentioned in
HIVE-25561 that if the speculatively executed task is killed, some data may be
submitted unexpectedly. However, after HIVE-25561, there is still a situation
that has not been solved. If two task attempts commit file at the same time,
the problem of duplicate data files may also occur. Although the probability of
this happening is very, very low, it does happen.
> Killed speculative execution task attempt should not commit file
> ----------------------------------------------------------------
>
> Key: HIVE-27899
> URL: https://issues.apache.org/jira/browse/HIVE-27899
> Project: Hive
> Issue Type: Bug
> Components: Tez
> Reporter: Chenyu Zheng
> Assignee: Chenyu Zheng
> Priority: Major
>
> As I mentioned in HIVE-25561, when tez turns on speculative execution, the
> data file produced by hive may be duplicated. I mentioned in HIVE-25561 that
> if the speculatively executed task is killed, some data may be submitted
> unexpectedly. However, after HIVE-25561, there is still a situation that has
> not been solved. If two task attempts commit file at the same time, the
> problem of duplicate data files may also occur. Although the probability of
> this happening is very, very low, it does happen.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
