Jim Halfpenny created NIFI-16611:
------------------------------------

             Summary: Add connect timeout to load balancing client sockets
                 Key: NIFI-16611
                 URL: https://issues.apache.org/jira/browse/NIFI-16611
             Project: Apache NiFi
          Issue Type: Bug
          Components: Core Framework
    Affects Versions: 2.12.0
            Reporter: Jim Halfpenny


{{{{{{NioAsyncLoadBalanceClient.createChannel()}}}}}} connects in blocking mode 
via {{socket.connect(address)}} without a timeout.  
{{nifi.cluster.load.balance.comms.timeout}} is applied with 
{{{}setSoTimeout{}}}, which does not cover connect. Against a peer that 
silently drops packets, for example the previous address of a restarted node in 
a dynamic environment such as Kubernetes, the call blocks for the OS TCP 
connect timeout (about 127 seconds on Linux with default settings).

{{NioAsyncLoadBalanceClientTask}} iterates serially over all registered 
clients. While one client is blocked in connect, load-balanced transfers to 
every other node served by that task thread are stalled. A failed connect only 
penalizes the client for 1 second, so the stall repeats for as long as the peer 
address is unreachable and the node is still considered connected.

This is the same failure mode as NIFI-16547 in the cluster protocol sockets.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to