Hello all,
I would like to ask you about the strange behavior that I am seeing when I am
using the vpp load-balancing plugin.
Below you can see the configuration that I am using inside the vpp:
vpp# set int ip addr HundredGigabitEthernet84/0/4 172.16.1.2/30
vpp# set int ip addr HundredGigabitEthernet84/9/1 192.168.1.215/24
vpp# set int state HundredGigabitEthernet84/9/1 up
vpp# set int state HundredGigabitEthernet84/0/4 up
lb vip 20.20.20.20/32 protocol udp port 1000 encap nat4 type nodeport
target_port 1000
lb as 20.20.20.20/32 protocol udp port 1000 172.16.1.1
lb set interface nat4 in HundredGigabitEthernet84/0/4
lb conf ip4-src-address 172.16.1.2
vpp# show lb vip verbose
ip4-nat4 [1] 20.20.20.20/32
new_size:1024
protocol:17 port:1000
type:nodeport port:59395 target_port:1000 counters:
packet from existing sessions: 0
first session packet: 0
untracked packet: 0
no server configured: 0
#as:1
172.16.1.1 1024 buckets 0 flows dpo:18 used
I am starting executing udp traffic towards the VIP address of the Load
balancer with the below netcat command:
nc -u 20.20.20.20 1000
The traffic reaches the receiver as you can see from the TCPdump output below
but it doesn’t get SNATed as it should be and as I configured it in vpp.
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on nsm0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:49:35.620264 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 8
14:49:36.975004 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 9
14:49:37.715221 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 7
14:49:39.010709 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 7
The Source Ip in the above tcpdump output should be 172.16.1.2 and not
192.168.1.106. Do you have any idea why this is happening ? Am I configuring
the vpp lb plugin incorrectly ? Attached you can find the vpp trace that is
captured when the traffic was flowing.
Thank you in advance,
Dimitris
{\rtf1\ansi\ansicpg1252\cocoartf2511
\cocoatextscaling0\cocoaplatform0{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
{\colortbl;\red255\green255\blue255;}
{\*\expandedcolortbl;;}
\paperw11900\paperh16840\margl1440\margr1440\vieww35640\viewh22400\viewkind0
\deftab720
\pard\pardeftab720\partightenfactor0
\f0\fs26 \cf0 \expnd0\expndtw0\kerning0
------------------- Start of thread 0 vpp_main -------------------\
No packets in trace buffer\
------------------- Start of thread 1 vpp_wk_0 -------------------\
Packet 1\
\'a0\
02:11:08:763316: dpdk-input\
\'a0 HundredGigabitEthernet84/0/4 rx queue 0\
\'a0 buffer 0xfed5b8: current data 0, length 80, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x1000000\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0l4-cksum-computed l4-cksum-correct\
\'a0\'a0PKT MBUF: port 0, nb_segs 1, pkt_len 80\
\'a0\'a0\'a0 buf_len 2176, data_len 80, ol_flags 0x80, data_off 128, phys_addr 0x3fb56e80\
\'a0\'a0\'a0 packet_type 0x691 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0\
\'a0\'a0\'a0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0\
\'a0\'a0\'a0 Packet Offload Flags\
\'a0\'a0\'a0\'a0\'a0 PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid\
\'a0\'a0\'a0 Packet Types\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L4_NONFRAG (0x0600) Non-fragmented IP packet\
\'a0 IP4: fa:6e:3d:37:9b:3f -> 6a:91:5f:b0:da:4c\
\'a0 ICMP: 172.16.1.1 -> 192.168.1.106\
\'a0\'a0\'a0 tos 0xc0, ttl 64, length 66, checksum 0x4bd6 dscp CS6 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0xbf01\
\'a0 ICMP destination_unreachable port_unreachable checksum 0x6c44\
02:11:08:763329: ethernet-input\
\'a0frame: flags 0x3, hw-if-index 1, sw-if-index 1\
\'a0 IP4: fa:6e:3d:37:9b:3f -> 6a:91:5f:b0:da:4c\
02:11:08:763341: ip4-input-no-checksum\
\'a0 ICMP: 172.16.1.1 -> 192.168.1.106\
\'a0\'a0\'a0 tos 0xc0, ttl 64, length 66, checksum 0x4bd6 dscp CS6 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0xbf01\
\'a0 ICMP destination_unreachable port_unreachable checksum 0x6c44\
02:11:08:763348: lb-nat4-in2out\
\'a0 lb vip[0]: ip6-gre6 ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff/0 new_size:1 #as:0\
lb as[0]: ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff removed\
lb nat: rx_sw_if_index = 1, next_index = 0\
02:11:08:763357: error-drop\
\'a0 rx:HundredGigabitEthernet84/0/4\
02:11:08:763359: drop\
\'a0 dpdk-input: no error\
\'a0\
Packet 2\
\'a0\
02:11:13:403106: dpdk-input\
\'a0 HundredGigabitEthernet84/0/4 rx queue 0\
\'a0 buffer 0xfed480: current data 0, length 60, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x1000001\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0l4-cksum-computed l4-cksum-correct\
\'a0\'a0PKT MBUF: port 0, nb_segs 1, pkt_len 60\
\'a0\'a0\'a0 buf_len 2176, data_len 60, ol_flags 0x0, data_off 128, phys_addr 0x3fb52080\
\'a0\'a0\'a0 packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0\
\'a0\'a0\'a0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0\
\'a0\'a0\'a0 Packet Types\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet\
\'a0 ARP: fa:6e:3d:37:9b:3f -> 6a:91:5f:b0:da:4c\
\'a0 request, type ethernet/IP4, address size 6/4\
\'a0 fa:6e:3d:37:9b:3f/172.16.1.1 -> 00:00:00:00:00:00/172.16.1.2\
02:11:13:403113: ethernet-input\
\'a0 frame: flags 0x3, hw-if-index 1, sw-if-index 1\
\'a0 ARP: fa:6e:3d:37:9b:3f -> 6a:91:5f:b0:da:4c\
02:11:13:403119: arp-input\
\'a0 request, type ethernet/IP4, address size 6/4\
\'a0 fa:6e:3d:37:9b:3f/172.16.1.1 -> 00:00:00:00:00:00/172.16.1.2\
02:11:13:403125: arp-reply\
\'a0 request, type ethernet/IP4, address size 6/4\
\'a0 fa:6e:3d:37:9b:3f/172.16.1.1 -> 00:00:00:00:00:00/172.16.1.2\
02:11:13:403138: HundredGigabitEthernet84/0/4-output\
\'a0 HundredGigabitEthernet84/0/4\
\'a0\'a0ARP: 6a:91:5f:b0:da:4c -> fa:6e:3d:37:9b:3f\
\'a0 reply, type ethernet/IP4, address size 6/4\
\'a0 6a:91:5f:b0:da:4c/172.16.1.2 -> fa:6e:3d:37:9b:3f/172.16.1.1\
02:11:13:403141: HundredGigabitEthernet84/0/4-tx\
\'a0 HundredGigabitEthernet84/0/4 tx queue 1\
\'a0 buffer 0xfed480: current data 0, length 60, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x1000001\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 l3-hdr-offset 14\
\'a0\'a0PKT MBUF: port 0, nb_segs 1, pkt_len 60\
\'a0\'a0\'a0 buf_len 2176, data_len 60, ol_flags 0x0, data_off 128, phys_addr 0x3fb52080\
\'a0\'a0\'a0 packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0\
\'a0\'a0\'a0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0\
\'a0\'a0\'a0 Packet Types\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet\
\'a0 ARP: 6a:91:5f:b0:da:4c -> fa:6e:3d:37:9b:3f\
\'a0 reply, type ethernet/IP4, address size 6/4\
\'a0 6a:91:5f:b0:da:4c/172.16.1.2 -> fa:6e:3d:37:9b:3f/172.16.1.1\
\'a0\
------------------- Start of thread 2 vpp_wk_1 -------------------\
Packet 1\
\'a0\
02:11:07:931410: dpdk-input\
\'a0 HundredGigabitEthernet84/9/1 rx queue 0\
\'a0 buffer 0xffac11: current data 0, length 60, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000000\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0l4-cksum-computed l4-cksum-correct\
\'a0\'a0PKT MBUF: port 1, nb_segs 1, pkt_len 60\
\'a0\'a0\'a0 buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 0x3feb04c0\
\'a0\'a0\'a0 packet_type 0x291 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0\
\'a0\'a0\'a0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0\
\'a0\'a0\'a0 Packet Offload Flags\
\'a0\'a0\'a0\'a0\'a0 PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid\
\'a0\'a0\'a0\'a0\'a0 PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid\
\'a0\'a0\'a0 Packet Types\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L4_UDP (0x0200) UDP packet\
\'a0 IP4: ea:49:9e:59:b4:14 -> 0a:5e:59:2e:55:db\
\'a0 UDP: 192.168.1.106 -> 20.20.20.20\
\'a0\'a0\'a0 tos 0x00, ttl 64, length 38, checksum 0x392b dscp CS0 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0x1762, flags DONT_FRAGMENT\
\'a0 UDP: 60058 -> 1000\
\'a0\'a0\'a0 length 18, checksum 0x2856\
02:11:07:931441: ethernet-input\
\'a0 frame: flags 0x3, hw-if-index 2, sw-if-index 2\
\'a0 IP4: ea:49:9e:59:b4:14 -> 0a:5e:59:2e:55:db\
02:11:07:931453: ip4-input-no-checksum\
\'a0 UDP: 192.168.1.106 -> 20.20.20.20\
\'a0\'a0\'a0 tos 0x00, ttl 64, length 38, checksum 0x392b dscp CS0 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0x1762, flags DONT_FRAGMENT\
\'a0 UDP: 60058 -> 1000\
\'a0\'a0\'a0 length 18, checksum 0x2856\
02:11:07:931462: ip4-lookup\
\'a0 fib 0 dpo-idx 0 flow hash: 0x00000000\
\'a0 UDP: 192.168.1.106 -> 20.20.20.20\
\'a0\'a0\'a0 tos 0x00, ttl 64, length 38, checksum 0x392b dscp CS0 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0x1762, flags DONT_FRAGMENT\
\'a0 UDP: 60058 -> 1000\
\'a0\'a0\'a0 length 18, checksum 0x2856\
02:11:07:931471: lb4-nat4-port\
\'a0 lb vip[1]: ip4-nat4 20.20.20.20/32 new_size:1024 #as:1\'a0 protocol:17 port:1000\'a0 type:nodeport port:59395 target_port:1000\
lb as[1]: 172.16.1.1 used\
\'a0\
02:11:07:931486: ip4-load-balance\
\'a0 fib 0 dpo-idx 2 flow hash: 0x00000000\
\'a0 UDP: 192.168.1.106 -> 172.16.1.1\
\'a0\'a0\'a0 tos 0x00, ttl 64, length 38, checksum 0xb441 dscp CS0 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0x1762, flags DONT_FRAGMENT\
\'a0 UDP: 60058 -> 1000\
\'a0\'a0\'a0 length 18, checksum 0xa36c\
02:11:07:931487: ip4-rewrite\
\'a0 tx_sw_if_index 1 dpo-idx 2 : ipv4 via 172.16.1.1 HundredGigabitEthernet84/0/4: mtu:9000 next:3 fa6e3d379b3f6a915fb0da4c0800 flow hash: 0x00000000\
\'a0 00000000: fa6e3d379b3f6a915fb0da4c080045000026176240003f11b541c0a8016aac10\
\'a0 00000020: 0101ea9a03e80012a36c6173646173646173640a0000000000000000\
02:11:07:931490: HundredGigabitEthernet84/0/4-output\
\'a0 HundredGigabitEthernet84/0/4\
\'a0\'a0IP4: 6a:91:5f:b0:da:4c -> fa:6e:3d:37:9b:3f\
\'a0 UDP: 192.168.1.106 -> 172.16.1.1\
\'a0\'a0\'a0 tos 0x00, ttl 63, length 38, checksum 0xb541 dscp CS0 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0x1762, flags DONT_FRAGMENT\
\'a0 UDP: 60058 -> 1000\
\'a0\'a0\'a0 length 18, checksum 0xa36c\
02:11:07:931492: HundredGigabitEthernet84/0/4-tx\
\'a0 HundredGigabitEthernet84/0/4 tx queue 2\
\'a0 buffer 0xffac11: current data 0, length 60, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000000\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 l3-hdr-offset 14\
\'a0\'a0PKT MBUF: port 1, nb_segs 1, pkt_len 60\
\'a0\'a0\'a0 buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 0x3feb04c0\
\'a0\'a0\'a0 packet_type 0x291 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0\
\'a0\'a0\'a0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0\
\'a0\'a0\'a0 Packet Offload Flags\
\'a0\'a0\'a0\'a0\'a0 PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid\
\'a0\'a0\'a0\'a0\'a0 PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid\
\'a0\'a0\'a0 Packet Types\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L4_UDP (0x0200) UDP packet\
\'a0 IP4: 6a:91:5f:b0:da:4c -> fa:6e:3d:37:9b:3f\
\'a0 UDP: 192.168.1.106 -> 172.16.1.1\
\'a0\'a0\'a0 tos 0x00, ttl 63, length 38, checksum 0xb541 dscp CS0 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0x1762, flags DONT_FRAGMENT\
\'a0 UDP: 60058 -> 1000\
\'a0\'a0\'a0 length 18, checksum 0xa36c\
\'a0\
Packet 2\
\'a0\
02:11:12:976226: dpdk-input\
\'a0 HundredGigabitEthernet84/9/1 rx queue 0\
\'a0 buffer 0xffaad9: current data 0, length 60, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000001\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
-- more -- (136-180/221)}
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17630): https://lists.fd.io/g/vpp-dev/message/17630
Mute This Topic: https://lists.fd.io/mt/77265184/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
