[
https://issues.apache.org/jira/browse/NIFI-16598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-16598.
-----------------------------------
Fix Version/s: 2.13.0
Resolution: Fixed
> Relationship retry can put FlowFile back on swap queue, changing queue order
> ----------------------------------------------------------------------------
>
> Key: NIFI-16598
> URL: https://issues.apache.org/jira/browse/NIFI-16598
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Wojciech Kochman
> Assignee: Wojciech Kochman
> Priority: Major
> Fix For: 2.13.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When a Processor routes a FlowFile to a retried relationship (for example
> failure with YIELD_PROCESSOR), StandardProcessSession returns the FlowFile to
> its original queue during commit. That requeue used FlowFileQueue.put() /
> putAll(). If the source queue is in swap mode, SwappablePriorityQueue appends
> the FlowFile to the swap queue instead of the active queue.
> The FlowFile was at the front of the active queue when it was polled, so it
> now sits behind every remaining active FlowFile. Later FlowFiles are
> processed first. This breaks ordering for any prioritizer, most noticeably
> First In First Out.
> Session rollback already avoids this path via FlowFileQueue.putToActive()
> (NIFI-16468). Relationship retry still went through the normal commit
> enqueue, so the same swap-mode reordering remains.
> This can be reproduced with a queue whose swap threshold is 10,000. Queue
> 10,001 FlowFiles, process the first one, and transfer it to a relationship
> configured for retry with YIELD_PROCESSOR. The next pull returns the second
> FlowFile instead of the first.
> To address this, use putToActive() from StandardProcessSession when
> committing a FlowFile whose destination is the same queue it was polled from
> (the retry path).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)