Hi,
I'm seeing an issue with DPDK and IP frag where the first chunk is getting
dropped on egress. I'm using the igb_e1000 driver with DPDK v18.x. My scenario
is that I'm allocating a new buffer to store an outer L3 tunnel header, I clone
the original received buffer which is a little over 3K, I link the newly cloned
buffer to the allocated buffers nextptr and copy the meta data from the
original packet buffer to the new allocated buffer. After calling
rte_ipv4_fragment_packet, I notice that the number of chunks created form the
jumbo packet seems correct. What's strange is that the 1st fragmented chunk has
a segment with a data_len = 0. All subsequent chunks appear to be fine. I've
ran rte_mbuf_sanity_check against my new packet which succeeds. After adjusting
the packet for L2 and adding the Ethernet header, I send the packet out but
only chunks 1 + 2 are received off the wire. The first chunk (0) is never
received and the NICs eth stats reflect this as well.
I'm hoping someone could shed some light on this issue and provide a solution.
Thanks,
Steve
Code example:
copybuf = rte_pktmbuf_alloc(getMbufPool());
// Add L3 tunnel header
Clone_buffer = rte_pktmbuf_clone(pkt_info->mblk, getClonePool());
// Link the outer trace header
copybuf->next = clone_buffer;
copybuf->pkt_len += clone_buffer->pkt_len; // update overall
packet length
clone_buffer->pkt_len = clone_buffer->data_len; // packet len only
in first segment
copybuf->nb_segs = (uint8_t)(clone_buffer->nb_segs + 1);
/* copy metadata from source packet */
copybuf->vlan_tci = pkt_info->mblk->vlan_tci;
copybuf->vlan_tci_outer = pkt_info->mblk->vlan_tci_outer;
copybuf->port = pkt_info->mblk->port;
copybuf->tx_offload = pkt_info->mblk->tx_offload;
copybuf->hash = pkt_info->mblk->hash;
copybuf->ol_flags = pkt_info->mblk->ol_flags;
rte_mbuf_sanity_check(copybuf, 1);
Rice# dump mbuf at 0x3100ae8e56c0, iova=42e8e5880, buf_len=2176
pkt_len=3687, ol_flags=182, nb_segs=4, in_port=0
segment at 0x3100ae8e56c0, data=0x3100ae8e5900, data_len=20
Dump data at [0x3100ae8e5900], len=20
00000000: 45 00 0E 67 00 00 00 00 80 04 82 0B C0 A8 94 6E | E..g...........n
00000010: C0 A8 94 C8 | | | | | | | | | | | | | ....
segment at 0x3100a6f319c0, data=0x3100a73b9b0e, data_len=1500
Dump data at [0x3100a73b9b0e], len=80
00000000: 45 00 0E 53 27 22 00 00 40 11 C4 DE AC 10 94 0A | E..S'"..@.......
00000010: AC 10 94 6E 13 C4 13 C4 0E 3F 00 1F 49 4E 56 49 | ...n.....?..INVI
00000020: 54 45 20 73 69 70 3A 73 65 72 76 69 63 65 40 31 | TE sip:service@1
00000030: 37 32 2E 31 36 2E 31 34 38 2E 31 31 30 3A 35 30 | 72.16.148.110:50
00000040: 36 30 20 53 49 50 2F 32 2E 30 0D 0A 56 69 61 3A | 60 SIP/2.0..Via:
segment at 0x3100a6f31900, data=0x3100a73b90a2, data_len=1480
segment at 0x3100a6f31840, data=0x3100a73b8622, data_len=687
Chunk 0: l2 len 0 l3 len 20, pkt len 1514, data len 34, nsegs 3
dump mbuf at 0x3100ae8e4c40, iova=42e8e4e00, buf_len=2176
pkt_len=1514, ol_flags=40000000000000, nb_segs=3, in_port=65535
segment at 0x3100ae8e4c40, data=0x3100ae8e4e72, data_len=34
Dump data at [0x3100ae8e4e72], len=34
00000000: 00 21 F6 69 02 A3 00 08 25 25 63 A4 08 00 45 00 | .!.i....%%c...E.
00000010: 05 DC FA CF 20 00 80 04 6F C6 C0 A8 94 6E C0 A8 | .... ...o....n..
00000020: 94 C8 | | | | | | | | | | | | | | | ..
segment at 0x3100ae8e41c0, data=0x3100ae8e5914, data_len=0
segment at 0x3100ae8e3740, data=0x3100a73b9b0e, data_len=1480
Dump data at [0x3100a73b9b0e], len=66
00000000: 45 00 0E 53 27 22 00 00 40 11 C4 DE AC 10 94 0A | E..S'"..@.......
00000010: AC 10 94 6E 13 C4 13 C4 0E 3F 00 1F 49 4E 56 49 | ...n.....?..INVI
00000020: 54 45 20 73 69 70 3A 73 65 72 76 69 63 65 40 31 | TE sip:service@1
00000030: 37 32 2E 31 36 2E 31 34 38 2E 31 31 30 3A 35 30 | 72.16.148.110:50
00000040: 36 30 | | | | | | | | | | | | | | | 60
Sending packet: txQ 1, L2 len 14, L3 len 20, protocol 4, flags 0 mtu 1500
egress intf 537919488
Chunk 1: l2 len 0 l3 len 20, pkt len 1514, data len 34, nsegs 3
dump mbuf at 0x3100ae8e2cc0, iova=42e8e2e80, buf_len=2176
pkt_len=1514, ol_flags=40000000000000, nb_segs=3, in_port=65535
segment at 0x3100ae8e2cc0, data=0x3100ae8e2ef2, data_len=34
Dump data at [0x3100ae8e2ef2], len=34
00000000: 00 21 F6 69 02 A3 00 08 25 25 63 A4 08 00 45 00 | .!.i....%%c...E.
00000010: 05 DC FA CF 20 B9 80 04 6F 0D C0 A8 94 6E C0 A8 | .... ...o....n..
00000020: 94 C8 | | | | | | | | | | | | | | | ..
segment at 0x3100ae8e2240, data=0x3100a73ba0d6, data_len=20
Dump data at [0x3100a73ba0d6], len=20
00000000: 37 32 2E 31 36 2E 31 34 38 2E 31 30 3A 35 30 36 | 72.16.148.10:506
00000010: 30 3E 3B 72 | | | | | | | | | | | | | 0>;r
segment at 0x3100ae8e17c0, data=0x3100a73b90a2, data_len=1460
Dump data at [0x3100a73b90a2], len=46
00000000: 65 61 73 6F 6E 3D 74 69 6D 65 2D 6F 66 2D 64 61 | eason=time-of-da
00000010: 79 0D 0A 44 69 76 65 72 73 69 6F 6E 3A 20 3C 73 | y..Diversion: <s
00000020: 69 70 3A 73 69 70 70 30 30 30 31 38 40 31 | | | ip:sipp00018@1
Sending packet: txQ 1, L2 len 14, L3 len 20, protocol 4, flags 0 mtu 1500
egress intf 537919488
Chunk 2: l2 len 0 l3 len 20, pkt len 741, data len 34, nsegs 3
dump mbuf at 0x3100ae8e0d40, iova=42e8e0f00, buf_len=2176
pkt_len=741, ol_flags=40000000000000, nb_segs=3, in_port=65535
segment at 0x3100ae8e0d40, data=0x3100ae8e0f72, data_len=34
Dump data at [0x3100ae8e0f72], len=34
00000000: 00 21 F6 69 02 A3 00 08 25 25 63 A4 08 00 45 00 | .!.i....%%c...E.
00000010: 02 D7 FA CF 01 72 80 04 91 59 C0 A8 94 6E C0 A8 | .....r...Y...n..
00000020: 94 C8 | | | | | | | | | | | | | | | ..
segment at 0x3100ae8e02c0, data=0x3100a73b9656, data_len=20
Dump data at [0x3100a73b9656], len=20
00000000: 6D 65 2D 6F 66 2D 64 61 79 0D 0A 44 69 76 65 72 | me-of-day..Diver
00000010: 73 69 6F 6E | | | | | | | | | | | | | sion
segment at 0x3100ae8df840, data=0x3100a73b8622, data_len=687
Dump data at [0x3100a73b8622], len=46
00000000: 3A 20 3C 73 69 70 3A 73 69 70 70 30 30 30 34 30 | : <sip:sipp00040
00000010: 40 31 37 32 2E 31 36 2E 31 34 38 2E 31 30 3A 35 | @172.16.148.10:5
00000020: 30 36 30 3E 3B 72 65 61 73 6F 6E 3D 74 69 | | | 060>;reason=ti
Sending packet: txQ 1, L2 len 14, L3 len 20, protocol 4, flags 0 mtu 1500
egress intf 537919488
Stephen Banville | Consulting Software Engineer
Phone: +1 781-538-7670
100 Crosby Drive | Bedford, MA 01730
CGBU DEV - Platform Engineering
https://oracle.zoom.us/my/steve.banville
_____
