markap14 commented on a change in pull request #4792:
URL: https://github.com/apache/nifi/pull/4792#discussion_r569490344
##########
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TailFile.java
##########
@@ -1184,7 +1191,15 @@ private boolean recoverRolledFiles(final ProcessContext
context, final ProcessSe
// This is the same file that we were reading when
we shutdown. Start reading from this point on.
rolledOffFiles.remove(0);
FlowFile flowFile = session.create();
- flowFile = session.importFrom(in, flowFile);
+ try {
+ ByteArrayOutputStream out = new
ByteArrayOutputStream();
Review comment:
This approach is going to buffer everything that gets read into the
ByteArrayOutputStream. This could potentially be a huge amount of data. We want
to be sure that we don't buffer that up. Instead, I would recommend an approach
like:
```
try (final OutputStream out = session.write(flowFile)) {
readLines(fis.getChannel(), ByteBuffer.allocate(65536), out, new CRC32(),
reReadOnNul, true);
}
```
This allows us to write directly to the content repository instead of
buffering the data in a ByteArrayOutputStream.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
