[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #2 from Richard Scheffenegger --- Can you please provide more context around the traces? Did you capture on the client side, or on the server side? It appears, that the capture was performed on the client side, with LRO enabled - simultaneously, the network in between appears to have created some significant reordering, causing the re-transmission and later DSACKs of packets by the sender. If the server is under your control, please create a simultaneous capture on both sides - and to narrow down things further, try to disable TSO / LRO (ifconfig eth0 -lro -tso ) on both sides. First things to establish: Is the sender transmitting these bytes incorrectly (with/without TSO)? Is the network interferring somehow? Is the clients LRO code mishandling the packets? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #3 from Keyong --- (In reply to Gleb Smirnoff from comment #1) The freeBSD deployed on the physical machine. The software used to send the data is "truenas". Thanks Keyong -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #4 from Keyong --- (In reply to Richard Scheffenegger from comment #2) The packet captured on both client and server. During the last reproduce this issue, the client/server LRO/TSO with default setting, I didn't change that. Then I disabled the DSACK and SACK in the server side(freeBSD), the client was a normal Centos VM. After disabled the DSACK and SACK I can't reproduce this issue with more than 20 times test. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #5 from Richard Scheffenegger --- Where are the two independent packet traces, if you traced on both client and server at the same time? Also, disabling SACK will lead to much worst packet loss recovery performance, while also prevent TSO / LRO issues simulatanouely. A SACK enabled, LRO/TSO disabled test would be interesting. I asked awk to create a simple 32bit sum of the 1448-byte payload segments making up the LRO chunks, for all the data segments with an even multiple: (sum32 is not perfect, but should be sufficient to see identical sub-sections; note that prior to the capture, the packets appear to have been sent before, which are in frame #4 - #21. the first 1448byte chunk in #63 is identical to #66, indicating this could be an off-by-one mbuf issue. All duplicate, subsequently seen chunks are prefixed with # below. packet #63 should have many overlaps with packets #55 and #57 - but they do NOT correspond properly... 3rd chunk of #63 should be 1st chunk of #55, but only chunks 4 and 2 (and following) are the same, similar between #63/13 and #57/1, where only subsequent chunks are the same... frame.number ip.src tcp.seq(rel) tcp.ack(rel) tcp.len #_1448_segs [sum32 sum32...] 2 10.234.1.9 35249 229 15928 11 72053392291 385714934591419632206729401031568921410043220839 398170924830381008236919388657879560400554947764390059647829 401051361480 4 10.234.1.9 8689 229 1448 1 402468043721 5 10.234.1.9 10137 229 1448 1 401277757279 6 10.234.1.9 11585 229 1448 1 415282924207 7 10.234.1.9 13033 229 1448 1 384078634026 8 10.234.1.9 14481 229 1448 1 403972324660 9 10.234.1.9 15929 229 1448 1 406377582180 10 10.234.1.9 17377 229 1448 1 395849724237 11 10.234.1.9 18825 229 1448 1 407105644009 12 10.234.1.9 20273 229 1448 1 391049306449 13 10.234.1.9 21721 229 1448 1 392972187658 14 10.234.1.9 23169 229 1448 1 412806674913 15 10.234.1.9 24617 229 1448 1 368815440121 16 10.234.1.9 26065 229 1448 1 441426216789 17 10.234.1.9 27513 229 1448 1 413962289118 18 10.234.1.9 28961 229 1448 1 410986024398 19 10.234.1.9 30409 229 1448 1 411104040736 20 10.234.1.9 31857 229 1448 1 401688330533 21 10.234.1.9 33305 229 1448 1 0 23 10.234.1.9 51177 229 2896 2 #0 203246954054 27 10.234.1.9 54073 229 5792 4 394852055780384030294443392529581011386143976667 29 10.234.1.9 59865 229 7240 5 396719530022411466448353385699980034379232637059398683035014 31 10.234.1.9 67105 229 2896 2 404469974242419769008759 33 10.234.1.9 70001 229 7240 5 390781603776405293380528401115408251408555360267401962764123 35 10.234.1.9 77241 229 5792 4 410414639507399252713459422519480994402128188056 37 10.234.1.9 83033 229 5792 4 397118187855389763261786385790875737396498279313 39 10.234.1.9 88825 229 10136 7 410833492195394727248034434370688504405654811556415670751773 381338791549363187342839 41 10.234.1.9 98961 229 7240 5 373714275313408842605624118929186736#0 772816864 43 10.234.1.9 106201 229 2896 2 409221826523405055397380 45 10.234.1.9 109097 229 5792 4 409248775808407813669100400196811736393952527415 47 10.234.1.9 114889 229 5792 4 428166500303407670732874391317321268398231044986 49 10.234.1.9 120681 229 4344 3 153542385751#0 1379131396 51 10.234.1.9 125025 229 7240 5 383437534358417050426910408105259593391024117815402553991311 55 10.234.1.9 132265 229 7240 5 398609113564394899607335404273196631377759935477372329922394 57 10.234.1.9 139505 229 17376 12 406899099155377008101980389519299155398503947093403573177749 412798843978378308644919409772644714397983176629424631857245 397732366425 386270375516 62 10.234.1.9 129369 229 1448 1 421647638868 63 10.234.1.9 129369 229 27512 19 409602081388400590437965395728025774#394899607335 #404273196631 #377759935477 #372329922394 407165511753411371394473384531841671 365127713737 40499019330440777546#377008101980 #389519299155 #398503947093 #403573177749 #412798843978 #378308644919 66 10.234.1.9 130817 457 1448 1 #409602081388 67 10.234.1.9 156881 457 5792 4 403565719067397983667401432116192336383972004598 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 Keyong changed: What|Removed |Added Attachment #235649|0 |1 is obsolete|| CC||keyong@smartx.com --- Comment #6 from Keyong --- Created attachment 237476 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=237476&action=edit client side packet trace Attached the client side packet trace. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #7 from Keyong --- Created attachment 237477 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=237477&action=edit Server side packet trace -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #8 from Keyong --- (In reply to Richard Scheffenegger from comment #5) Thanks for your detail analyze, sorry to forget attach the client side packet trace, attached both sides now. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #9 from Richard Scheffenegger --- ClientSide: 2 10.234.1.9 35249 229 11584 8 72053392291 385714934591419632206729401031568921410043220839 398170924830381008236919388657879560 3 10.234.1.9 46833 229 4344 3 403699027891383855600553414816586574 6 10.234.1.9 1 1 8688 6 397048024999401090144041391424219729389140047353373127431023 386317540465 10 10.234.1.9 8689 229 4344 3 402468043721422687157409399988543889 12 10.234.1.9 13033 229 21720 15 384078634026408436223210400564575027378555159070406385296146 401195322034395907015544389483782230409510812379434652283767 392616332722 413309374917394133931772418769171848410069530380 14 10.234.1.9 51177 229 2896 2 0 203246954054 16 10.234.1.9 54073 229 4344 3 394852055780384030294443392529581011 17 10.234.1.9 58417 229 1448 1 420918862786 20 10.234.1.9 59865 229 4344 3 396719530022411466448353385699980034 21 10.234.1.9 64209 229 2896 2 398567707744401151853126 24 10.234.1.9 67105 229 2896 2 404469974242419769008759 26 10.234.1.9 70001 229 7240 5 390781603776405293380528401115408251408555360267401962764123 27 10.234.1.9 77241 229 4344 3 410414639507399252713459422519480994 28 10.234.1.9 81585 229 1448 1 441132535691 32 10.234.1.9 83033 229 5792 4 397118187855389763261786385790875737396498279313 34 10.234.1.9 88825 229 5792 4 410833492195394727248034434370688504405654811556 35 10.234.1.9 94617 229 4344 3 414298751919397266345619427889638344 36 10.234.1.9 98961 229 7240 5 373714275313408842605624118929186736#0 772816864 37 10.234.1.9 106201 229 2896 2 409221826523405055397380 42 10.234.1.9 109097 229 2896 2 409248775808407813669100 43 10.234.1.9 111993 229 2896 2 403633422329414484235525 46 10.234.1.9 114889 229 5792 4 428166500303407670732874391317321268398231044986 47 10.234.1.9 120681 229 4344 3 153542385751#0 1379131396 48 10.234.1.9 125025 229 4344 3 383437534358417050426910408105259593 50 10.234.1.9 132265 229 2896 2 398609113564394899607335 51 10.234.1.9 135161 229 4344 3 372905179588407165511753411371394473 55 10.234.1.9 139505 229 13032 9 406899099155377008101980389519299155398503947093403573177749 412798843978378308644919409772644714397983176629 57 10.234.1.9 152537 229 4344 3 383951095350386259078212401351762973 59 10.234.1.9 129369 229 1448 1 421647638868 60 10.234.1.9 129369 229 27512 19 409602081388400590437965395728025774#394899607335 404273196631 377759935477372329922394#407165511753 #411371394473 384531841671 365127713737 40499019330440777546#377008101980 #389519299155 #398503947093 #403573177749 #412798843978 #378308644919 61 10.234.1.9 130817 457 1448 1 #409602081388 62 10.234.1.9 156881 457 5792 4 403565719067397983667401432116192336383972004598 63 10.234.1.9 162673 457 2896 2 382446051083448088161130 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588
--- Comment #10 from Richard Scheffenegger ---
There seems to be a discrepancy in the TSO vs LRO payload bytes (maybe my
script is broken?):
Server:
2 10.234.1.9 35249 229 15928 11
72053392291 385714934591419632206729401031568921410043220839
398170924830381008236919388657879560400554947764
390059647829401051361480
Client:
2 10.234.1.9 35249 229 11584 8
72053392291 385714934591419632206729401031568921410043220839
398170924830381008236919388657879560
3 10.234.1.9 46833 229 4344 3
403699027891383855600553414816586574
The last 3 chunks from the server side trace, don't seem to be identical with
the three chunks received in a separate LRO aggregation (!).
This is how I create these chunk-sums:
tshark -r ClientIdenticalSeq.pcap -T fields -e frame.number -e ip.src -e
tcp.seq -e tcp.ack -e tcp.len -e tcp.options.sack_le -e tcp.options.sack_re -e
tcp.payload | awk '
//{
fr=int($5/1448);
if ((fr*1448 == $5) && (fr > 0)) {
print $1" "$2" "$3" "$4" "$5" "fr;
for (i=0;i 1) { printf("#") };
printf("%d \t", s);
}
}
printf("\n");
}
My expectation would be that the sums calculated this way for the tcp payload
is identical when TSO / LRO capture data...
Note that this difference is visible before the SACK retransmissions.
Was the capture created just before issuing the NFS read? Because the
expectation there would be for the sender side to transmit all the data in
sequence, and not skip over some data. It appears the captures were trimmed
after taking, in the middle of a loss recovery episode from a prior NFS read
IO...
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 256820] netinet6: etherip over ipv6 doesn't work
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256820 Koichiro Iwao changed: What|Removed |Added CC||h...@freebsd.org --- Comment #8 from Koichiro Iwao --- MFC'd into stable/13 and stable/12. Thanks hps for the help! BTW, what about stable/11? AFAIK, stable/11 branch is not EoL'ed yet. However, 11.4 is EoL and 11.5 will probably never be released so I personally have less motivation to MFC it into stable/11 branch. May I close this now? Or should I close after MFC into stable/11? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #11 from Keyong --- (In reply to Richard Scheffenegger from comment #10) Yes, the capture created before issuing the read. Because the issue was not reproduced 100%, so need to capture with a long time, the origin packet was too big, and then filter with the "V4 Reply (Call In 1) READ[Illegal Segments]" -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 Hans Petter Selasky changed: What|Removed |Added CC||hsela...@freebsd.org --- Comment #12 from Hans Petter Selasky --- Are you using the latest Linux kernel? I noticed something strange when copying files to a Linux box using scp recently (I'm using 14-main GENERIC-NODEBUG) and the problems recently disappeared. Did you check the Linux tcp commits recently? For example this one: commit 686dc2db2a0fdc1d34b424ec2c0a735becd8d62b Author: Neal Cardwell Date: Sat Sep 3 08:10:23 2022 -0400 tcp: fix early ETIMEDOUT after spurious non-SACK RTO Fix a bug reported and analyzed by Nagaraj Arankal, where the handling of a spurious non-SACK RTO could cause a connection to fail to clear retrans_stamp, causing a later RTO to very prematurely time out the connection with ETIMEDOUT. Here is the buggy scenario, expanding upon Nagaraj Arankal's excellent report: https://github.com/LineageOS/android_kernel_xiaomi_onclite/commit/be8176c25c32607cf357317880038a1e4a935bd7 --HPS -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #13 from Keyong --- (In reply to Hans Petter Selasky from comment #12) Thanks for your information. The client Linux kernel was not the lastest. With the SACK disabled I haven't reproduce this issue. Only can reproduce this issue with SACK/DSACK enabled. Will double check the Linux tcp commits. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 Michael Tuexen changed: What|Removed |Added CC||tue...@freebsd.org --- Comment #14 from Michael Tuexen --- Now I'm confused... The server running FreeBSD is sending a stream of bytes and the client is receiving them. The server should behave consistent, no matter whether the peer behaves correctly or not. So why does it matter if the peer has a bug? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #15 from Michael Tuexen --- Is it possible to test if the problem does still exists in stable/13? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #16 from Richard Scheffenegger --- My script was not using correctly calculating the index when extracting the payload bytes (vs. nibbles). Server: 2 10.234.1.9 35249 229 15928 11 0x71d1c76e360xb735f725570xbad512b35a0xc83acec2650xb844d58b55 0xbd5a0399800xb4bfc3dab30xbee06a47f30xbfb010681c0xb6b3fa5ac5 0x4f9fb874c4 4 10.234.1.9 8689 229 1448 1 0xbab2063451 5 10.234.1.9 10137 229 1448 1 0xbbf86dbb36 6 10.234.1.9 11585 229 1448 1 0xb8ec7aad26 7 10.234.1.9 13033 229 1448 1 0xc2229a8c78 8 10.234.1.9 14481 229 1448 1 0xbc727adfaf 9 10.234.1.9 15929 229 1448 1 0xb878e4e750 10 10.234.1.9 17377 229 1448 1 0xb8cf084795 11 10.234.1.9 18825 229 1448 1 0xb1c0a9f0b7 12 10.234.1.9 20273 229 1448 1 0xbd4ea6026b 13 10.234.1.9 21721 229 1448 1 0xbb4ccfd34f 14 10.234.1.9 23169 229 1448 1 0xbbef592031 15 10.234.1.9 24617 229 1448 1 0xc11f3fa432 16 10.234.1.9 26065 229 1448 1 0xc17d2e2122 17 10.234.1.9 27513 229 1448 1 0xb9c4b6b54b 18 10.234.1.9 28961 229 1448 1 0xafdab1ffd9 19 10.234.1.9 30409 229 1448 1 0xba2bb58c02 20 10.234.1.9 31857 229 1448 1 0x85d214ca1a 21 10.234.1.9 33305 229 1448 1 0x5d03fe02 23 10.234.1.9 51177 229 2896 2 0x2e276667310xb9fa1ff668 27 10.234.1.9 54073 229 5792 4 0xbeb29edeb30xc8c8f9156e0xba5c9bda1c0xb926f50a14 29 10.234.1.9 59865 229 7240 5 0xb80428de210xbc989742af0xb7d87eb4900xc0764c96580xb0f453f54f 31 10.234.1.9 67105 229 2896 2 0xbf11b17dd60xb7ae02bd2e 33 10.234.1.9 70001 229 7240 5 0xbcace865e10xa94237837a0xb9cc730xc378ca422f0xae8c77d34c 35 10.234.1.9 77241 229 5792 4 0xb9aa1f5cf00xbc0d4d5eeb0xbfcfd22bb50xbe5f4fcab3 37 10.234.1.9 83033 229 5792 4 0xb38cf37f360xb8411349490xb254f6cd100xb9308b33c0 39 10.234.1.9 88825 229 10136 7 0xb7c4d84be00xb756bea53a0xc01788147f0xbe0c0c256b0xbfd5dbc475 0xba1280798a0xbb7f22a3b0 41 10.234.1.9 98961 229 7240 5 0xc0d849757c0x1e0bb5fc960xe103fe02 0xa981db3da50xb2ea9bfc15 43 10.234.1.9 106201 229 2896 2 0xb804c256b20xadee432a00 45 10.234.1.9 109097 229 5792 4 0xbb18c54acf0xad6fd40bd10xb5b26d9bee0xc2241f2310 47 10.234.1.9 114889 229 5792 4 0xb06dc4a15f0xbb4bffa5a30xc0be16026c0xb66fde201b 49 10.234.1.9 120681 229 4344 3 0x297e5fe5310x4e68b0fd4b0xb9d403d1b5 51 10.234.1.9 125025 229 7240 5 0xb23ee730dd0xc2fd29b2eb0xbe27bbf1be0xb829e51fdc0xbc5b20682b 55 10.234.1.9 132265 229 7240 5 0xbc514488230xb84ce9af5d0xb9d8c684d10xbdf8798dd20xb8e9ab3099 57 10.234.1.9 139505 229 17376 12 0xbb89e633050xc39d87e5390xbe3907a9120xc1d4e2a4cd0xb67ad48c56 0xb8a34776090xb910bcd6580xc22c316cbc0xb2ed5e07d30xc29ab7b16b 0xbb7fbe80df 0xb31f21cf40 62 10.234.1.9 129369 229 1448 1 #0xb829e51fdc 63 10.234.1.9 129369 229 27512 19 #0xbc5b20682b #0xbc51448823 #0xb84ce9af5d #0xb9d8c684d1 #0xbdf8798dd2 #0xb8e9ab3099 #0xbb89e63305 #0xc39d87e539 #0xbe3907a912 #0xc1d4e2a4cd#0xb67ad48c56#0xb8a3477609 #0xb910bcd658 #0xc22c316cbc #0xb2ed5e07d3 #0xc29ab7b16b #0xbb7fbe80df #0xb31f21cf40 0xbf0ecdd485 66 10.234.1.9 130817 457 1448 1 #0xbc5b20682b 67 10.234.1.9 156881 457 5792 4 #0xbf0ecdd485 0xba2a5a7f850xb1861afe360xb12d2968f0 Client: 2 10.234.1.9 35249 229 11584 8 0x71d1c76e360xb735f725570xbad512b35a0xc83acec2650xb844d58b55 0xbd5a0399800xb4bfc3dab30xbee06a47f3 3 10.234.1.9 46833 229 4344 3 0xbfb010681c0xb6b3fa5ac50x4f9fb874c4 6 10.234.1.9 1 1 8688 6 0xb4745a9cce0xbad4180f420xb0272a08680xbfb7166be90xb89d58abc8 0xbe04548606 10 10.234.1.9 8689 229 4344 3 0xbab20634510xbbf86dbb360xb8ec7aad26 12 10.234.1.9 13033 229 21720 15 0xc2229a8c780xbc727adfaf0xb878e4e7500xb8cf0847950xb1c0a9f0b7 0xbd4ea6026b0xbb4ccfd34f0xbbef5920310xc11f3fa4320xc17d2e2122 0xb9c4b6b54b 0xafdab1ffd90xba2bb58c020x85d214ca1a0x5d03fe02 14 10.234.1.9 51177 229 2896 2 0x2e276667310xb9fa1ff668 16 10.234.1.9 54073 229 4344 3 0xbeb29edeb30xc8c8f9156e0xba5c9bda1c 17 10.234.1.9 58417 229 1448 1 0xb926f50a14 20 10.234.1.9 59865 229 4344 3 0xb80428de210xbc989742af0xb7d87eb490 21 10.234.1.9 64209 229 2896 2 0xc0764c96580xb0f453f54f 24 10.234.1.9 67105 229 2896 2 0xbf11b17dd60xb7ae02bd2e 26 10.234.1.9 70001 229 7240 5 0xbcace865e10xa94237837a0xb9cc730xc378ca422f0xae8c77d34c 27 10.234.1.9 77241 229 4344 3 0xb9aa1f5cf00xbc0d4d5eeb0xbfcfd22bb5 28 10.234.1.9 81585 229 1448 1 0xbe5f4fcab3 32 10.234.1.9 83033 229 5792 4 0xb38cf37f360xb8411349490xb254f6cd100xb9308b33c0 34 10.234.1.9 88825 229 5792 4 0xb7c4d84be00xb756bea53a0xc01788147f0xbe0c0c256b 35 10.234.1.9 94617 229 4344 3 0xbfd5dbc4750xba1280798a0xbb7f22a3b0 36 10.234.1.9 98961 229 7240 5 0xc0d8
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #17 from Richard Scheffenegger --- 1448 byte chunks between server and client align perfectly until server frame #51 (5 chunks: 0xb23ee730dd0xc2fd29b2eb0xbe27bbf1be0xb829e51fdc 0xbc5b20682b) and client frame #48 (3 chunks 0xb23ee730dd0xc2fd29b2eb 0xbe27bbf1be). The last two chunks of the TSO get dropped. server #62 is the lost 4th chunk of #51. And server #63 starts with chunk 5 of #51 but doesn't have the correct sequence number! Frame 66 repeats chunk 5 of #51, with the correct sequence number. TSO frame 63 seems to send everything from snd.una for an entire window, but the seq number is off by one segment (1448) to the left. However, 19 packets during SACK loss recovery is unexpected too! (In short, I feel like the TSO frame #63 shouldn't be there at all). I wonder if this may be the issue https://reviews.freebsd.org/D36637 although that's dealing with the traffic burst, not the erraneous sequence number... -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 Richard Scheffenegger changed: What|Removed |Added Status|New |Open -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588
--- Comment #18 from Richard Scheffenegger ---
For completeness, the awk script with proper indexing:
cat output | awk '
//{
fr=int($5/1448);
if ((fr*1448 == $5) && (fr > 0)) {
print $1" "$2" "$3" "$4" "$5" "fr;
gsub("[^0-9a-fA-F]", "", $6);
for (i=0;i 1) { printf("#") };
printf("0x%08x\t", s);
}
}
printf("\n");
}
'
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 Michael Zhilin changed: What|Removed |Added CC||miz...@freebsd.org Severity|Affects Only Me |Affects Some People --- Comment #19 from Michael Zhilin --- Hi, I've found same defect on 13.1-RELEASE with fetching of packages via HTTP protocol. Shortly speaking, https://reviews.freebsd.org/D36046 is fix for this bug. It's already committed to stable/13 and main. The issue is not reproduced on 14-current and stable/13. My setup is following: - FreeBSD guest in data center in same town (with poudriere and HTTP nginx) - I use VPN & WiFi to access it Sometimes packets are lost and server may start retransmission of several packets with same payload but with different sequence numbers. Sometimes, but not always. Then it sends same payload third time with correct payload. :/ For testing I download 500MiB file 4 times: fetch -q http://192.168.20.12:8000/gcc11-11.3.0_1.pkg && md5 gcc11-11.3.0_1.pkg fetch -q http://192.168.20.12:8000/gcc11-11.3.0_1.pkg && md5 gcc11-11.3.0_1.pkg fetch -q http://192.168.20.12:8000/gcc11-11.3.0_1.pkg && md5 gcc11-11.3.0_1.pkg fetch -q http://192.168.20.12:8000/gcc11-11.3.0_1.pkg && md5 gcc11-11.3.0_1.pkg If all checksums are correct, kernel is good. I've bisected it to commit: https://github.com/freebsd/freebsd-src/commit/c21b7b55bea2cc2bf3b420c70a9018e703ed6f00 commit c21b7b55bea2cc2bf3b420c70a9018e703ed6f00 Author: Richard Scheffenegger Date: Wed Aug 31 14:49:25 2022 +0200 tcp: finish SACK loss recovery on sudden lack of SACK blocks While a receiver should continue sending SACK blocks for the duration of a SACK loss recovery, if for some reason the TCP options no longer contain these SACK blocks, but we already started maintaining the Scoreboard, keep on handling incoming ACKs (without SACK) as belonging to the SACK recovery. Reported by:thj Reviewed by:tuexen, #transport MFC after: 2 weeks Sponsored by: NetApp, Inc. Differential Revision: https://reviews.freebsd.org/D36046 sys/netinet/tcp_input.c | 39 ++- 1 file changed, 22 insertions(+), 17 deletions(-) Many thanks to Gleb and Richard! -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #20 from Richard Scheffenegger --- This behavior was likely introduced by 0533fab89e37caab886568df88d9f939e85eeb6c D28634 (Mar 25 2021), but addressed with c21b7b55bea2cc2bf3b420c70a9018e703ed6f00 D36046 (Aug 31 2022). Looking through the commit log, it seems that 13.0 is not affected, only 13.1 is affected. stable/13 and the upcoming 13.2 won't be affected. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 Richard Scheffenegger changed: What|Removed |Added Status|Open|In Progress Priority|--- |Normal Severity|Affects Some People |Affects Many People -- You are receiving this mail because: You are the assignee for the bug.
[Bug 265588] [TCP] - tcp send a retransmission identical sequence number packet with different payload
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265588 --- Comment #21 from Keyong --- Many thanks to all provide helps! -- You are receiving this mail because: You are the assignee for the bug.
