> On 9. Dec 2024, at 10:50, Pavel Vazharov <pa...@x3me.net> wrote:
>
> Hi there,
>
> We are using the network stack of FreeBSD 13 on top of DPDK in our
> application.
> During the last tests in the lab I stumbled upon the following situation:
> 1. It's a test where 5000 parallel connections are opened by Apache
> Bench and each one downloads 1MB data. It causes the client NIC to
> start dropping packets due to overflows which is intentional behavior.
> 2. The server side is our application with the FreeBSD stack. The
> client side Ubuntu 24.04 with Linux 6.8.0.
> 3. So, a connection is opened and the download starts on it. At some
> point the first drops occur and according to the TCP dump, from the
> client side, they take a few seconds before the connection heals up.
> However, these drops lead to increased values of t_srtt, t_rttvar and
> thus to increased value of t_rxtcur.
Do you observe increased values of t_rxtcur due to exponential backoff
or due to extreme values of t_srtt and t_rttvar?
>
> 4. The window opens again up to 100-200 KB with lots of packets
> in-flight and the drops start again. They cause the re-transmit timer
> from the FreeBSD side to be started but with an interval of something
> like 18-20 seconds (according to my printf debugging on this side).
> 5. At the same time the TCP keep-alive timer is also started for the
> same connection (it's enabled for all connections) with a timeout of
> 15 seconds.
> 6. Nothing happens on this connection for the next 15 seconds. I'm not
> sure why the Linux stack didn't send any "wake-up" ACK packets or
> something but the tcpdump from the client side shows full silence
> between 14-th and 29-th second.
> 7. Next the FreeBSD keep-alive logic kicks-in and sends an ACK packet
> which is ACK-ed by the Linux stack immediately. However, this ACK
> packet received by the FreeBSD stack leads to restart of the
> retransmit timer and with the interval which is bigger than the
> keep-alive interval.
> 8. Point 6 and 7 repeat one more time before the apache bench client
> gives up on this connection and declares that it's timed-out. My
> understanding is that the connection can "loop" in 6-7 for a very long
> time and a packet with data will never be retransmitted.
Can you provide a .pcap file?
Best regards
Michael
> 9. As far as I debugged the situation from the FreeBSD side the
> restart of the retransmit timer happens in the code after the
> `process_ACK` label, in the else branch here:
> ```
> if (th->th_ack == tp->snd_max) {
> tcp_timer_activate(tp, TT_REXMT, 0);
> needoutput = 1;
> } else if (!tcp_timer_active(tp, TT_PERSIST))
> tcp_timer_activate(tp, TT_REXMT, tp->t_rxtcur);
> ```
>
> So, based on the above situation I've the following questions:
> 1. Would it be correct if the re-transmit timer is not restarted by
> keep-alive ACK packets?
> 2. Assuming that the above change won't break anything else, is there
> a way for detecting that an ACK packet acknowledges previously sent
> keep-alive packet?
>
> Regards,
> Pavel.
>
