[
https://issues.apache.org/jira/browse/HADOOP-19251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874238#comment-17874238
]
Steve Loughran commented on HADOOP-19251:
-----------------------------------------
yeah. you try to have one unified FS with mountpoints underneath,
hasPathCapability() takes a path and so is designed for virtual filesystems.
but it doesn't let you say "can I do an atomic file/dir rename between these
two paths"
why not start a [DISCUSS] conversation on hadoop-common dev and including
hdfs-dev for their opinions, as it is the hdfs implementation which is normative
> Add Options.Rename.THROW_NON_ATOMIC
> -----------------------------------
>
> Key: HADOOP-19251
> URL: https://issues.apache.org/jira/browse/HADOOP-19251
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 3.3.6
> Reporter: Alkis Evlogimenos
> Priority: Major
>
> I propose we add an option `Options.Rename.THROW_NON_ATOMIC` to change
> `rename()` behavior to throw when the underlying filesystem's rename
> operation is not atomic.
> This would be useful for callers that expect to perform an atomic op but want
> to fail if when an atomic rename fails.
>
> At first this might seem something that can be done by querying capabilities
> of the filesystem but that would only work on real filesystems. A motivating
> example would be a virtual filesystem for which paths can resolve to any
> concrete filesystem (s3, etc). If `rename()` is called with two virtual paths
> that resolve to different filesystems (s3 and gcs for example) then obviously
> the operation can't be atomic since bytes must be copied from one fs to
> another.
>
> What do you think [~steve_l] ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
