[
https://issues.apache.org/jira/browse/HIVE-28822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28822:
--------------------------------
Description:
Concurrent `INSERT INTO` statements that create the **same new dynamic
partition** on an external non-ACID Hive table backed by S3 can fail with
`MoveTask return code 40000`. The root cause is a check-then-rename race in
`Hive.mvFile`'s `_copy_N` naming loop: two writers pick the same
destination path between an `exists()` probe and the subsequent
`rename()` call, and the loser gets a `FileAlreadyExistsException` from
`S3AFileSystem.initiateRename`.
This is separate from HIVE-29744, which addresses the *silent* data-loss
twin of this race (concurrent writers overwriting each other with no
error). This ticket is specifically about the *fail-loud* variant that
survives even after HIVE-29744.
was:TODO: add description later
> Concurrent INSERTs into a new non-ACID dynamic partition fail with
> FileAlreadyExistsException on S3
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-28822
> URL: https://issues.apache.org/jira/browse/HIVE-28822
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
>
> Concurrent `INSERT INTO` statements that create the **same new dynamic
> partition** on an external non-ACID Hive table backed by S3 can fail with
> `MoveTask return code 40000`. The root cause is a check-then-rename race in
> `Hive.mvFile`'s `_copy_N` naming loop: two writers pick the same
> destination path between an `exists()` probe and the subsequent
> `rename()` call, and the loser gets a `FileAlreadyExistsException` from
> `S3AFileSystem.initiateRename`.
> This is separate from HIVE-29744, which addresses the *silent* data-loss
> twin of this race (concurrent writers overwriting each other with no
> error). This ticket is specifically about the *fail-loud* variant that
> survives even after HIVE-29744.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
