Hi Vijay,
When you create an ip-ip tunnel the source address is the one that is local to
the device on which you are configuring the tunnel. The destination address is
that of the peer. Given packets arrive at your device with address 20.20.99.99,
you tunnel addresses are backwards.
/neale
From: Vijay Kumar <vjkumar2...@gmail.com>
Date: Monday, 8 March 2021 at 10:20
To: Neale Ranns <ne...@graphiant.com>
Subject: Re: [vpp-dev] Traffic is not put on IPSec tunnel intf ipip0
Hi Neale,
The ipip tunnel src points to the VTH IP hosted in VPP while ipip tunnel dst
points to an interface hosted in the peer VM (Strongswan). The ESP pkt in the
trace has source ip that points to the IP of Strongswan.
I have pasted the output of trace and show ipip. The IPs in the pkt trace
output is opposite (reversed) to the IPs in the show ipip tunnel cmd. This
could be because my traffic direction right (SS is starting traffic)? Or is
this a problem
NOTE: -
I created ipip interface and installed SA using the same logic as in
ikev2_add_tunnel_from_main()
vpp#show trace
00:04:10:067403: dpdk-input
VirtualFuncEthernet0/6/0 rx queue 0
buffer 0x4c3000: current data 0, length 170, buffer-pool 0, ref-count 1,
totlen-nifb 0, trace handle 0x1000000
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 170
buf_len 2176, data_len 170, ol_flags 0x180, data_off 128, phys_addr
0x50ac0080
packet_type 0x691 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without
extension headers
RTE_PTYPE_L4_NONFRAG (0x0600) Non-fragmented IP packet
IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:9a:2c:38 802.1q vlan 1556
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0x5757 dscp CS0 ecn NON_ECN
fragment id 0xf37a, flags DONT_FRAGMENT
00:04:10:067423: ethernet-input
frame: flags 0x3, hw-if-index 3, sw-if-index 3
IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:9a:2c:38 802.1q vlan 1556
00:04:10:067429: ip4-input
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0x5757 dscp CS0 ecn NON_ECN
fragment id 0xf37a, flags DONT_FRAGMENT
00:04:10:067435: ip4-lookup
fib 1 dpo-idx 24 flow hash: 0x00000000
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0x5757 dscp CS0 ecn NON_ECN
fragment id 0xf37a, flags DONT_FRAGMENT
00:04:10:067439: ip4-local
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0x5757 dscp CS0 ecn NON_ECN
fragment id 0xf37a, flags DONT_FRAGMENT
00:04:10:067440: ip4-punt
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0x5757 dscp CS0 ecn NON_ECN
fragment id 0xf37a, flags DONT_FRAGMENT
00:04:10:067443: error-punt
rx:VirtualFuncEthernet0/6/0.1556
00:04:10:067448: punt
ip4-local: unknown ip protocol
vpp# show ipip tun
tunnel tunnel-hash
vpp# show ipip tunnel
[0] instance 0 src 20.20.99.99 dst 20.20.99.215 table-ID 0 sw-if-idx 18 flags
[none] dscp CS0
vpp#
On Mon, Mar 8, 2021 at 2:31 PM Neale Ranns
<ne...@graphiant.com<mailto:ne...@graphiant.com>> wrote:
Hi Vijay,
VPP drops because the packets don’t classify to your tunnel, not because ESP is
not registered. Compare the addresses in the output from ‘sh ipip tun’ with the
packet in the trace.
/neale
From: Vijay Kumar <vjkumar2...@gmail.com<mailto:vjkumar2...@gmail.com>>
Date: Monday, 8 March 2021 at 09:52
To: Neale Ranns <ne...@graphiant.com<mailto:ne...@graphiant.com>>
Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] Traffic is not put on IPSec tunnel intf ipip0
Hi Neale,
Yes, you are right.
This is the output of show ip local cmd. The IPSEC_ESP is not registered. Hence
the drop.
NOTE: -
===========
I have a working POC setup that has IKEv2 plugin enabled. In this POC setup, I
can find IPSEC_ESP in the show ip local output.
On my setup, I disabled the plugin ikev2. I only have a vnet ipsec graph node
(but no ikev2). The ike stack is running on a separate VM which is that of
Azure ike stack.
vpp# show ip local
Protocols handled by ip4_local
ICMP: ip4-icmp-input
IGMP: igmp-input
IP_IN_IP: ipip4-input
TCP: an_ppe_router_input
UDP: ip4-udp-lookup
IPV6: ipip4-input
VRRP: vrrp4-input
unknown 200: an_ppe_wfectrl
unknown 201: an_ppe_vppctrl_input
vpp#
vpp#
On Mon, Mar 8, 2021 at 2:08 PM Neale Ranns
<ne...@graphiant.com<mailto:ne...@graphiant.com>> wrote:
Hi Vijay,
‘unknown IP protocol’ means there is no registered handler for ESP. From the
info you gave I am assuming you are using ip-ip tunnel protection with the SAs
in transport mode. In that case the incoming packet should classify to the
ip-ip tunnel. The fact that it doesn’t, and the fact that the per through the
tunnel is unreachable, suggests that the src,dst addresses on this tunnel are
incorrect. The VPP IKE plugin would have set these addresses correctly for you
based on the IKE session end point.
/neale
From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
<vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> on behalf of Vijay Kumar via
lists.fd.io<http://lists.fd.io>
<vjkumar2003=gmail....@lists.fd.io<mailto:gmail....@lists.fd.io>>
Date: Monday, 8 March 2021 at 08:44
To: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] Traffic is not put on IPSec tunnel intf ipip0
Hi,
Below is the trace output for "unknown ip protocol" related drops for the ESP
pkts coming from the strongswan to VPP.
Let me know If I need to add some ip4-punt related config.
show trace o/p
===============
00:08:29:451365: dpdk-input
VirtualFuncEthernet0/7/0 rx queue 0
buffer 0x4c2d69: current data 0, length 170, buffer-pool 0, ref-count 1,
totlen-nifb 0, trace handle 0x1000001
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 170
buf_len 2176, data_len 170, ol_flags 0x180, data_off 128, phys_addr
0x4ecb5ac0
packet_type 0x691 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without
extension headers
RTE_PTYPE_L4_NONFRAG (0x0600) Non-fragmented IP packet
IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:c2:b4:f4 802.1q vlan 1556
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0xbe54 dscp CS0 ecn NON_ECN
fragment id 0x8c7d, flags DONT_FRAGMENT
00:08:29:451372: ethernet-input
frame: flags 0x3, hw-if-index 3, sw-if-index 3
IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:c2:b4:f4 802.1q vlan 1556
00:08:29:451377: ip4-input
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0xbe54 dscp CS0 ecn NON_ECN
fragment id 0x8c7d, flags DONT_FRAGMENT
00:08:29:451380: ip4-lookup
fib 1 dpo-idx 27 flow hash: 0x00000000
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0xbe54 dscp CS0 ecn NON_ECN
fragment id 0x8c7d, flags DONT_FRAGMENT
00:08:29:451382: ip4-local
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0xbe54 dscp CS0 ecn NON_ECN
fragment id 0x8c7d, flags DONT_FRAGMENT
00:08:29:451383: ip4-punt
IPSEC_ESP: 20.20.99.215 -> 20.20.99.99
tos 0x00, ttl 64, length 152, checksum 0xbe54 dscp CS0 ecn NON_ECN
fragment id 0x8c7d, flags DONT_FRAGMENT
00:08:29:451384: error-punt
rx:VirtualFuncEthernet0/7/0.1556
00:08:29:451386: punt
ip4-local: unknown ip protocol
On Mon, Mar 8, 2021 at 2:00 AM Vijay Kumar via lists.fd.io<http://lists.fd.io>
<vjkumar2003=gmail....@lists.fd.io<mailto:gmail....@lists.fd.io>> wrote:
Hi,
Strongswan <======================> [VPP IPSec] + [Azure IKE stack]
I have a use case like above. Strongswan is the peer while VPP is the
responder. On the right hand side, I am not showing VPP ikev2 plugin because we
have disabled the plugin but rather using the Microsoft Azure ike stack. In our
case the ike stack and vpp run on different pods. We have written a new plugin
for IKE pkt forwarding to the pod where IKE stack is running. The new plugin
registers for udp port 500 and 4500 and thus fwds all the received IKE traffic
to IKE stack (to be precise, the new VPP plugin forwards IKE packets to the
application process via memif. The application process in turn uses REST API to
forward the IKE pkts to the pods that runs the IKE stack). Once SA is
established, the IKE stack would install the SA on to the VPP IPSec plugin
using VPP APIs (we would use the vl_api_rpc_call_main_thread function call to
execute the REST API SA information).
The IPSec SA is established fine as displayed below but ping to the host on the
strongswan side is failing as "blackholed packets"
CLIs logs
================
vpp# show ipsec sa
[0] sa 256349522 (0xf479552) spi 3475331116 (0xcf25582c) protocol:esp
flags:[anti-replay ]
[1] sa 256349523 (0xf479553) spi 305419899 (0x1234567b) protocol:esp
flags:[anti-replay inbound ]
vpp#
NOTE
=============
20.200.75.1 is the host on the strongswan side, while
VirtualFuncEthernet0/7/0.1556 is the interface that is connected to the
Strongswan (this interface is pingable with the SS interface)
vpp# show interface
Name Idx State MTU (L3/IP4/IP6/MPLS) Counter
Count
VirtualFuncEthernet0/7/0 3 up 1500/0/0/0 rx packets
12
rx bytes
2432
tx packets
12
tx bytes
2408
drops
8
VirtualFuncEthernet0/7/0.1556 14 up 1500/0/0/0 rx packets
30
rx bytes
647112
tx packets
12
tx bytes
2408
drops
8
ip4
8
VirtualFuncEthernet0/9/0 4 down 9000/0/0/0
ipip0 18 down 9000/0/0/0
local0 0 down 0/0/0/0
vpp# set interface ip address ipip0 20.200.76.2/32<http://20.200.76.2/32>
vpp# set interface state ipip0 up
vpp# ip route add 20.200.75.1/32<http://20.200.75.1/32> via ipip0
vpp# set interface unnumbered ipip0 use VirtualFuncEthernet0/7/0.1556
vpp# show interface
Name Idx State MTU (L3/IP4/IP6/MPLS)
Counter Count
VirtualFuncEthernet0/7/0 3 up 1500/0/0/0 rx packets
14
rx bytes
2618
tx packets
14
tx bytes
2594
drops
9
VirtualFuncEthernet0/7/0.1556 14 up 1500/0/0/0 rx packets
34
rx bytes
711766
tx packets
14
tx bytes
2594
drops
9
ip4
9
VirtualFuncEthernet0/9/0 4 down 9000/0/0/0
ipip0 18 up 9000/0/0/0
local0 0 down 0/0/0/0
ISSUE
=========
All the ping traffic from SS are coming to VPP as ESP packets but the VPP is
dropping them as "unknown ip protocol". Also, for all the ping packets
triggered from VPP side are not being put onto the ipip0 IPSec tunnel but they
are dropped as "blackholed packets"
NOTE: -
==============
On the VPP, after adding a route to the destination 20.200.75.1, the show ip
FIB output shows dpo-drop rather than send via "VirtualFuncEthernet0/7/0.1556"
as shown below.
vpp# show if fib
20.200.75.1/32<http://20.200.75.1/32>
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:81 buckets:1 uRPF:109 to:[0:0]]
[0] [@6]: ipv4 via 0.0.0.0 ipip0: mtu:9000 next:16
450000000000000040048b9814146363141463d7
stacked-on entry:78:
[@1]: dpo-drop ip4
vpp# show node counters
Count Node Reason
10 null-node blackholed packets
81 memif-input not ip packet
2731 an_ppe_wfectrl wfectrl packets received
2731 an_ppe_wfectrl wfectrl replies sent
2181 an_ppe_wfectrl session stat request received
1 an_ppe_wfectrl service construct config
request received
1 an_ppe_wfectrl service construct config
request success
1 an_ppe_wfectrl service config request
received
1 an_ppe_wfectrl service config request success
453 an_ppe_wfectrl dpi stats request received
453 an_ppe_wfectrl dpi stats request success
19 an_ppe_wfectrl nat stats request received
19 an_ppe_wfectrl nat stats request success
58 arp-reply ARP replies sent
81 ip4-udp-lookup No error
14 ip4-local unknown ip protocol
vpp#
vpp#
NOTE
=============
When I did a POC using VPP ikev2 plugin and VPP IPSec, I did not face this
issue. The ping traffic from SS would be successfully decrypted at VPP and also
the reply sent back to SS. In the POC, the route entry of the destination was
fine. It was not dpo-drop.
Please suggest if there is any way to debug the traffic drops as "Unknown ip
protocol" for incoming ESP traffic and traffic drops as "blackholed packets"
for outgoing traffic
Any help is highly appreciated.
Regards,
Vijay Kumar N
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18885): https://lists.fd.io/g/vpp-dev/message/18885
Mute This Topic: https://lists.fd.io/mt/81158278/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
