mosermw commented on code in PR #9595:
URL: https://github.com/apache/nifi/pull/9595#discussion_r2012825191
##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PackageFlowFile.java:
##########
@@ -120,17 +122,34 @@
})
public class PackageFlowFile extends AbstractProcessor {
+ public static final String BATCHING_BEHAVIOUR_DESCRIPTION = """
+ "Maximum Batch Size" > 1 can improve storage or transmission
efficiency by batching many FlowFiles together into 1 larger file.
+ "Maximum Batch Content Size" can be used to enforce a soft
upper limit on the overall package size.
+
+ Note, that the Batch properties only restrict the maximum
amount of FlowFiles to incorporate into a single package.
+ In case less FlowFiles are queued than the properties allow
for,
+ the processor will not wait for the limits to be reached but
create smaller packages instead.
+ """;
+
public static final PropertyDescriptor BATCH_SIZE = new
PropertyDescriptor.Builder()
.name("max-batch-size")
.displayName("Maximum Batch Size")
- .description("Maximum number of FlowFiles to package into one
output FlowFile using a best effort, non guaranteed approach."
- + " Multiple input queues can produce unexpected batching
behavior.")
+ .description("Maximum number of FlowFiles to package into one
output FlowFile.")
.required(true)
.defaultValue("1")
.addValidator(StandardValidators.createLongValidator(1, 10_000,
true))
.build();
- private static final List<PropertyDescriptor> PROPERTIES =
List.of(BATCH_SIZE);
+ public static final PropertyDescriptor BATCH_CONTENT_SIZE = new
PropertyDescriptor.Builder()
+ .name("Maximum Batch Content Size")
+ .description("Maximum combined content size of FlowFiles to
package into one output FlowFile. " +
+ "Note, that FlowFiles whose content exceeds this limit are
packaged separately.")
+ .required(true)
+ .defaultValue("1 GB")
Review Comment:
After thinking about this more, and considering that this property enforces
a soft upper limit on packaged file size, a large value here won't affect
anything if Maximum Batch Size = 1. So no changes needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
