Hi Ole, -l option worked. I set it to 16 rather than the default 8 KB.
I actually came to this problem while trying to figure out another issue (I don't know if it's related). I had a chance to test my VPP setup using an ixia box. For some reason, ixia test was working only in one direction (sending packets from 10.155.6.x to 10.155.3.x) But while sending in the opposite direction, both TCP and UDP tests failed. Packets are being dropped by VPP. Packet capture showed these two outputs: error-drop ip4-input: ip4 length > l2 length error-drop, null-node: blackholed packets Here is my vpp.conf and packet capture: https://gist.github.com/ironpillow/9b1c5dd0905135ff09eba6067db179ae - Do you think the issues are related? - Also, how can I reproduce this with iperf? Thanks for taking the time. On Mon, Mar 4, 2019 at 12:00 PM Ole Troan <otr...@employees.org> wrote: > > > Got it. > > > > Since I wanted to test both upstream and downstream with iperf3, I was > > using -R option. > > Even with disabling virtual-reassembly, packets are being dropped (see > > below). > > > > Switching server to 10.155.6.x and client on 10.155.3.x works. > > So, for this kind of test, do you recommend switching client and > > server subnets instead of running iperf3 with -R. > > I’m recommending not triggering fragmentation. > Fragments are NAT unfriendly. > iperf with -l being sensible, as in not 8KB for TCP. Unless you have an 8KB+ > MTU. > > The iperf authors have clearly not read our draft in intarea. ;-) > > Cheers, > Ole > > > > > > here are the dropped packets: > > > > Packet 49 > > > > 00:19:41:823757: dpdk-input > > GigabitEthernet5/0/0 rx queue 0 > > buffer 0x2dd18: current data 0, length 834, free-list 0, clone-count > > 0, totlen-nifb 0, trace 0x30 > > ext-hdr-valid > > l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 > > PKT MBUF: port 3, nb_segs 1, pkt_len 834 > > buf_len 2176, data_len 834, ol_flags 0x180, data_off 128, > > phys_addr 0xe8b74680 > > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > > Packet Offload Flags > > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > > Packet Types > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > > UDP: 10.155.6.111 -> 10.155.3.201 > > tos 0x00, ttl 64, length 820, checksum 0x843f > > fragment id 0xd06f offset 7400, flags > > UDP: 23507 -> 4311 > > length 43450, checksum 0x6383 > > 00:19:41:823759: ethernet-input > > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > > 00:19:41:823760: l2-input > > l2-input: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > > 00:19:41:823760: l2-learn > > l2-learn: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > > bd_index 5 > > 00:19:41:823760: l2-fwd > > l2-fwd: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > > bd_index 5 > > 00:19:41:823761: ip4-input > > UDP: 10.155.6.111 -> 10.155.3.201 > > tos 0x00, ttl 64, length 820, checksum 0x843f > > fragment id 0xd06f offset 7400, flags > > UDP: 23507 -> 4311 > > length 43450, checksum 0x6383 > > 00:19:41:823761: nat44-in2out > > NAT44_IN2OUT_FAST_PATH: sw_if_index 19, next index 4, session -1 > > 00:19:41:823761: nat44-in2out-reass > > NAT44_REASS: sw_if_index 19, next index 1, status translated > > 00:19:41:823762: error-drop > > nat44-in2out-reass: Drop fragment > > > > Packet 50 > > > > 00:19:41:824581: dpdk-input > > GigabitEthernet5/0/0 rx queue 0 > > buffer 0x1b3a8: current data 0, length 1514, free-list 0, > > clone-count 0, totlen-nifb 0, trace 0x31 > > ext-hdr-valid > > l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 > > PKT MBUF: port 3, nb_segs 1, pkt_len 1514 > > buf_len 2176, data_len 1514, ol_flags 0x180, data_off 128, > > phys_addr 0xe86cea80 > > packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 > > rss 0x0 fdir.hi 0x0 fdir.lo 0x0 > > Packet Offload Flags > > PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid > > PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid > > Packet Types > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > > UDP: 10.155.6.111 -> 10.155.3.201 > > tos 0x00, ttl 64, length 1500, checksum 0x8700 > > fragment id 0xaea3, flags MORE_FRAGMENTS > > UDP: 5201 -> 47346 > > length 8200, checksum 0x5570 > > 00:19:41:824582: ethernet-input > > IP4: c0:56:27:11:f2:ac -> de:ad:00:00:00:05 > > 00:19:41:824583: l2-input > > l2-input: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > > 00:19:41:824583: l2-learn > > l2-learn: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > > bd_index 5 > > 00:19:41:824584: l2-fwd > > l2-fwd: sw_if_index 4 dst de:ad:00:00:00:05 src c0:56:27:11:f2:ac > > bd_index 5 > > 00:19:41:824584: ip4-input > > UDP: 10.155.6.111 -> 10.155.3.201 > > tos 0x00, ttl 64, length 1500, checksum 0x8700 > > fragment id 0xaea3, flags MORE_FRAGMENTS > > UDP: 5201 -> 47346 > > length 8200, checksum 0x5570 > > 00:19:41:824584: nat44-in2out > > NAT44_IN2OUT_FAST_PATH: sw_if_index 19, next index 4, session -1 > > 00:19:41:824585: nat44-in2out-reass > > NAT44_REASS: sw_if_index 19, next index 1, status translated > > 00:19:41:824585: error-drop > > nat44-in2out-reass: Drop fragment > > > > Thanks > > > > On Sun, Mar 3, 2019 at 11:35 PM Ole Troan <otr...@employees.org> wrote: > >> > >> Hi Carlito, > >> > >> Seems like you are sending IP fragments. > >> Those need to be (virtually) reassembled before NATted. Reassembly is to a > >> large extent an attack vector, and it’s rate limited. > >> > >> cheers, > >> Ole > >> > >>> On 3 Mar 2019, at 22:46, carlito nueno <carlitonu...@gmail.com> wrote: > >>> > >>> Hi all, > >>> > >>> While running more iperf3 udp tests, I noticed vpp status showing this: > >>> > >>> My current vpp conf: > >>> https://gist.github.com/ironpillow/4b119b57e21b31a7ff6985bcb20f952b > >>> > >>> Setup to reproduce: > >>> 1. iperf3 server on 10.155.3.2 (iperf3 -s -B 10.155.3.2) > >>> 2. iperf3 client on 10.155.6.2 but with -R flag (iperf3 -B 10.155.6.2 > >>> -c 10.155.3.2 -u -b0 -R) > >>> > >>> So basically, server on one subnet and client on another subnet and > >>> run it with -R flag > >>> > >>> vpp.service - vector packet processing engine > >>> Loaded: loaded (/lib/systemd/system/vpp.service; enabled; vendor > >>> preset: enabled) > >>> Active: active (running) since Fri 2019-03-01 17:09:24 PST; 18min ago > >>> Process: 32079 ExecStopPost=/bin/rm -f /dev/shm/db > >>> /dev/shm/global_vm /dev/shm/vpe-api (code=exited, status=0/SUCCESS) > >>> Process: 32093 ExecStartPre=/sbin/modprobe uio_pci_generic > >>> (code=exited, status=0/SUCCESS) > >>> Process: 32087 ExecStartPre=/bin/rm -f /dev/shm/db > >>> /dev/shm/global_vm /dev/shm/vpe-api (code=exited, status=0/SUCCESS) > >>> Main PID: 32095 (vpp_main) > >>> Tasks: 10 (limit: 4915) > >>> CGroup: /system.slice/vpp.service > >>> └─32095 /usr/bin/vpp -c /etc/vpp/startup.conf > >>> > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: no free resassembly slot > >>> Mar 01 17:20:17 test1 vnet[32095]: nat: --- message(s) throttled --- > >>> > >>> Thanks > >>> -=-=-=-=-=-=-=-=-=-=-=- > >>> Links: You receive all messages sent to this group. > >>> > >>> View/Reply Online (#12410): https://lists.fd.io/g/vpp-dev/message/12410 > >>> Mute This Topic: https://lists.fd.io/mt/30206523/675193 > >>> Group Owner: vpp-dev+ow...@lists.fd.io > >>> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [otr...@employees.org] > >>> -=-=-=-=-=-=-=-=-=-=-=- > >> > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > > > View/Reply Online (#12419): https://lists.fd.io/g/vpp-dev/message/12419 > > Mute This Topic: https://lists.fd.io/mt/30206523/675193 > > Group Owner: vpp-dev+ow...@lists.fd.io > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [otr...@employees.org] > > -=-=-=-=-=-=-=-=-=-=-=- >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12421): https://lists.fd.io/g/vpp-dev/message/12421 Mute This Topic: https://lists.fd.io/mt/30206523/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
