[
https://issues.apache.org/jira/browse/NIFI-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17459333#comment-17459333
]
ASF subversion and git services commented on NIFI-9260:
-------------------------------------------------------
Commit 534e6eafe7b8622e6d141802a40d7c6e01a09361 in nifi's branch
refs/heads/support/nifi-1.15 from Bence Simon
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=534e6ea ]
NIFI-9260 Making the 'write and rename' behaviour optional for PutHDFS
This closes #5423.
Signed-off-by: Peter Turcsanyi <[email protected]>
> Making "write and rename" behaviour optional in PutHDFS
> -------------------------------------------------------
>
> Key: NIFI-9260
> URL: https://issues.apache.org/jira/browse/NIFI-9260
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Simon Bence
> Assignee: Simon Bence
> Priority: Major
> Fix For: 1.16.0, 1.15.1
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> In some cases the current "write and rename" behaviour of PutHDFS
> undesirable. Particularly if the the server is backed with AWS S3 which has
> no "native" rename but copies file content when renaming, serious perfomance
> issues might arise.
> In order to avoid this, I propose to make this behaviour optional and make it
> possible for the PutHDFS to write directly to the final destination. Of
> course, depending on the use case, there is the risk of reading
> "half-written" files in this manner, but in some scenarios this is not a
> danger. Also, in case of S3 this should not be an issue as files in S3 are
> visible only after the "write" operation is finished
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
