[
https://issues.apache.org/jira/browse/NIFI-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-16599.
-------------------------------------
Fix Version/s: 2.13.0
Resolution: Fixed
> Preserve FlowFile order across restart for queues that are swapping
> -------------------------------------------------------------------
>
> Key: NIFI-16599
> URL: https://issues.apache.org/jira/browse/NIFI-16599
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Fix For: 2.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When a connection holds more FlowFiles than its swap threshold, FlowFiles can
> come out of the queue in the wrong order after NiFi restarts. There are three
> causes.
> If more FlowFiles are restored from the FlowFile Repository than the active
> queue can hold, but there are no swap files, whichever FlowFiles are restored
> first fill the active queue. The restore order is arbitrary, so newer
> FlowFiles can be processed ahead of older ones. NIFI-16580 fixed this only
> for the case where swap files exist.
> A selective drop rewrites a swap file under a new name that uses the current
> time. On restart, swap files are sorted by the time in their names, so the
> rewritten file is placed behind swap files that hold newer FlowFiles.
> Swap files written within the same millisecond are ordered by a random UUID
> in their names, so their order after a restart is random.
> SwappablePriorityQueue should restore FlowFiles in priority order when there
> are no swap files. FileSystemSwapManager should name swap files so that a
> rewritten file keeps its place and files written in the same millisecond sort
> in the order they were written.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)