[
https://issues.apache.org/jira/browse/MINIFICPP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gábor Gyimesi updated MINIFICPP-1967:
-------------------------------------
Description: When TailFile starts to process a new file it processes the
whole file with all its lines in a single onTrigger call. Due to this if a
large file is presented it results in lot of memory usage for a short time to
process all the flow files at once. We should add an option to maximize the
number of flow files generated in a single onTrigger call to lower the memory
consumption at a single point in time. (was: When TailFile starts to process a
new file it processes the whole file with all its lines in a single onTrigger
call. Due to this if a large file is presented it results in lot of memory
usage for a short time to process all the flow files at once. We should add an
option to maximize the number of flow files generated in a single onTrigger
call to lower the memory consumption at a single point of time.)
> Add batch processing of lines in TailFile
> -----------------------------------------
>
> Key: MINIFICPP-1967
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1967
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
> Reporter: Gábor Gyimesi
> Priority: Major
>
> When TailFile starts to process a new file it processes the whole file with
> all its lines in a single onTrigger call. Due to this if a large file is
> presented it results in lot of memory usage for a short time to process all
> the flow files at once. We should add an option to maximize the number of
> flow files generated in a single onTrigger call to lower the memory
> consumption at a single point in time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
