[
https://issues.apache.org/jira/browse/NIFI-1322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634244#comment-15634244
]
ASF GitHub Bot commented on NIFI-1322:
--------------------------------------
GitHub user pvillard31 opened a pull request:
https://github.com/apache/nifi/pull/1181
NIFI-1322 - PutHDFS - allow file append resolution
Two remarks:
- LocalFileSystem object used for unit tests does not implement the append
method.
- Based on HDFS-4600 (https://issues.apache.org/jira/browse/HDFS-4600),
appending to a file is only possible if the replication factor is lesser or
equal than the number of data nodes in the cluster. For live testing with a
standalone instance, consider changing the replication factor to 1 for the file
you are appending to.
Also, I considered that it would be up to the user to handle the demarcator
between two appended messages if such a demarcator is expected.
Finally, if two processors are appending to the same file, errors may be
raised due to concurrent access to the file. However, if redirecting the
failure relationship back to the processor, the processors should, in the end,
be able to append the messages. Obviously, in such a case, message ordering
shall not be a requirement. Local testing suggests that users should use a
MergeContent processor as upstream processor in order to ensure acceptable
bandwidth.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pvillard31/nifi NIFI-1322
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1181.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1181
----
commit a42ed1d547762b51dbe27275aa54e0736e7046d4
Author: Pierre Villard <[email protected]>
Date: 2016-11-02T19:38:52Z
PutHDFS - allow file append resolution
----
> Support appending files in PutHDFS
> ----------------------------------
>
> Key: NIFI-1322
> URL: https://issues.apache.org/jira/browse/NIFI-1322
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Scott
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
