[
https://issues.apache.org/jira/browse/NIFI-14386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17938766#comment-17938766
]
ASF subversion and git services commented on NIFI-14386:
--------------------------------------------------------
Commit 87e92122fc9d18747a828d6e66e50df4bd05820d in nifi's branch
refs/heads/main from Michael W Moser
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=87e92122fc ]
NIFI-14386 Remove ListenHTTP Max Data to Receive per Second property and
document the replacement Use Case (#9823)
Signed-off-by: David Handermann <[email protected]>
> Remove ListenHTTP Max Data to Receive per Second capability
> -----------------------------------------------------------
>
> Key: NIFI-14386
> URL: https://issues.apache.org/jira/browse/NIFI-14386
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Michael W Moser
> Assignee: Michael W Moser
> Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ListenHTTP can limit the rate of data that it accepts, which limits the rate
> that data is read from a socket. It will eventually read all data from the
> socket and commit the file to NiFi repositories. However, perhaps due to
> data buffering, the sending side can think it has sent the entire file but
> get a SocketTimeoutException because ListenHTTP hasn't finished reading yet.
> Typically, after encountering an exception, the sending side can try to send
> the file again. This causes duplicate files on the receiving end.
> Additionally, large files may never make it through the connection. Because
> of this we should remove the Max Data to Receive per Second capability.
> There is a viable replacement which should be documented. Follow ListenHTTP
> with a ControlRate processor where you set the desired data rate limit.
> Configure back pressure on the "success" connection between ListenHTTP and
> ControlRate. Back pressure should be limited to an acceptable data buffer
> size that is not too large such that the rate limit is unnoticeable.
> When back pressure is in effect, ListenHTTP will send 503 Service Unavailable
> responses as soon as an HTTP connection is made. This indicates to the
> sender that the ListenHTTP is overloaded and to try again later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
